[==========] Running 3 tests from 1 test case. [----------] Global test environment set-up. [----------] 3 tests from ClientFailoverOnNegotiationTimeoutITest [ RUN ] ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer I0507 20:05:14.457260 211 external_mini_cluster.cc:786] Running /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-master /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-master --fs_wal_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/wal --fs_data_dirs=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data --block_manager=log --webserver_interface=localhost --ipki_ca_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.0.52.254:52118 --webserver_interface=127.0.52.254 --webserver_port=0 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/logs --server_dump_info_path=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --raft_heartbeat_interval_ms=25 --leader_failure_exp_backoff_max_delta_ms=1000 --rpc_reuseport=true with env {} WARNING: Logging before InitGoogleLogging() is written to STDERR W0507 20:05:14.651139 217 flags.cc:412] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0507 20:05:14.651402 217 flags.cc:412] Enabled unsafe flag: --never_fsync=true W0507 20:05:14.653898 217 flags.cc:412] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000 W0507 20:05:14.653983 217 flags.cc:412] Enabled experimental flag: --ipki_ca_key_size=1024 W0507 20:05:14.654070 217 flags.cc:412] Enabled experimental flag: --ipki_server_key_size=1024 W0507 20:05:14.654104 217 flags.cc:412] Enabled experimental flag: --tsk_num_rsa_bits=512 W0507 20:05:14.654151 217 flags.cc:412] Enabled experimental flag: --rpc_reuseport=true I0507 20:05:14.658833 217 master_main.cc:66] Master server non-default flags: --leader_failure_exp_backoff_max_delta_ms=1000 --raft_heartbeat_interval_ms=25 --fs_data_dirs=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data --fs_wal_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/wal --ipki_ca_key_size=1024 --ipki_server_key_size=1024 --tsk_num_rsa_bits=512 --rpc_bind_addresses=127.0.52.254:52118 --rpc_reuseport=true --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data/info.pb --webserver_interface=127.0.52.254 --webserver_port=0 --never_fsync=true --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --log_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/logs --logbuflevel=-1 --logtostderr=true Master server version: kudu 1.8.0-cdh5.16.0-SNAPSHOT revision 4253c68ab2140f5294ab236d1c0fc445ad80f766 build type FASTDEBUG built by jenkins at 07 May 2018 12:28:50 PST on quasar-snqeym-1.vpc.cloudera.com build id 50748 ASAN enabled I0507 20:05:14.665354 217 master_main.cc:73] Initializing master server... I0507 20:05:14.666203 217 system_ntp.cc:143] NTP initialized. Skew: 500ppm Current error: 1158388us I0507 20:05:14.667794 217 fs_manager.cc:260] Metadata directory not provided I0507 20:05:14.667868 217 fs_manager.cc:266] Using write-ahead log directory (fs_wal_dir) as metadata directory I0507 20:05:14.668949 217 server_base.cc:433] Could not load existing FS layout: Not found: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data/instance: No such file or directory (error 2) I0507 20:05:14.669039 217 server_base.cc:434] Attempting to create new FS layout instead I0507 20:05:14.673254 217 fs_manager.cc:595] Generated new instance metadata in path /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data/instance: uuid: "476aed1601be4605b69e0622e3eeb7fc" format_stamp: "Formatted at 2018-05-07 20:05:14 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:14.673847 217 fs_manager.cc:595] Generated new instance metadata in path /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/wal/instance: uuid: "476aed1601be4605b69e0622e3eeb7fc" format_stamp: "Formatted at 2018-05-07 20:05:14 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:14.682034 217 fs_manager.cc:495] Time spent creating directory manager: real 0.008s user 0.004s sys 0.004s I0507 20:05:14.683228 217 env_posix.cc:1643] Not raising this process' open files per process limit of 65536; it is already as high as it can go I0507 20:05:14.683840 217 file_cache.cc:470] Constructed file cache lbm with capacity 26214 I0507 20:05:14.687862 217 fs_manager.cc:417] Time spent opening block manager: real 0.002s user 0.000s sys 0.004s I0507 20:05:14.687932 217 fs_manager.cc:428] Opened local filesystem: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data,/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/wal uuid: "476aed1601be4605b69e0622e3eeb7fc" format_stamp: "Formatted at 2018-05-07 20:05:14 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:14.688100 217 fs_report.cc:347] Block manager report -------------------- 1 data directories: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0507 20:05:14.724335 217 env_posix.cc:1643] Not raising this process' running threads per effective uid limit of 60284; it is already as high as it can go I0507 20:05:14.725477 217 master_main.cc:76] Starting Master server... I0507 20:05:14.739137 217 rpc_server.cc:205] RPC server started. Bound to: 127.0.52.254:52118 I0507 20:05:14.739672 217 webserver.cc:173] Starting webserver on 127.0.52.254:0 I0507 20:05:14.739702 217 webserver.cc:184] Document root disabled I0507 20:05:14.740448 217 webserver.cc:311] Webserver started. Bound to: http://127.0.52.254:59487/ I0507 20:05:14.741226 217 server_base.cc:610] Dumped server information to /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/master-0/data/info.pb I0507 20:05:14.745649 276 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:14.748805 211 external_mini_cluster.cc:848] Started /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-master as pid 217 I0507 20:05:14.749958 211 external_mini_cluster.cc:786] Running /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/wal --fs_data_dirs=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data --block_manager=log --rpc_bind_addresses=127.0.52.193:0 --local_ip_for_outbound_sockets=127.0.52.193 --webserver_interface=127.0.52.193 --webserver_port=0 --tserver_master_addrs=127.0.52.254:52118 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/logs --server_dump_info_path=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --raft_heartbeat_interval_ms=25 --leader_failure_exp_backoff_max_delta_ms=1000 --heartbeat_interval_ms=25 with env {} I0507 20:05:14.761056 276 tablet_bootstrap.cc:436] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Bootstrap starting. I0507 20:05:14.764206 276 tablet_bootstrap.cc:587] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: No blocks or log segments found. Creating new log. I0507 20:05:14.765180 276 log.cc:520] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Log is configured to *not* fsync() on all Append() calls I0507 20:05:14.767009 276 tablet_bootstrap.cc:436] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: No bootstrap required, opened a new log I0507 20:05:14.774475 276 raft_consensus.cc:303] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476aed1601be4605b69e0622e3eeb7fc" member_type: VOTER } I0507 20:05:14.774755 276 raft_consensus.cc:329] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:14.774806 276 raft_consensus.cc:604] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 476aed1601be4605b69e0622e3eeb7fc, State: Initialized, Role: FOLLOWER I0507 20:05:14.775486 276 consensus_queue.cc:227] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476aed1601be4605b69e0622e3eeb7fc" member_type: VOTER } I0507 20:05:14.775727 276 raft_consensus.cc:343] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0507 20:05:14.775805 276 raft_consensus.cc:435] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0507 20:05:14.775883 276 raft_consensus.cc:2704] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Snoozing failure detection for 0.109s (starting election) I0507 20:05:14.775975 276 raft_consensus.cc:2756] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:14.777693 276 raft_consensus.cc:457] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476aed1601be4605b69e0622e3eeb7fc" member_type: VOTER } I0507 20:05:14.778163 276 leader_election.cc:257] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I0507 20:05:14.778997 279 raft_consensus.cc:2704] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Snoozing failure detection for 0.086s (election complete) I0507 20:05:14.779122 279 raft_consensus.cc:2511] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 1 FOLLOWER]: Leader election won for term 1 I0507 20:05:14.795123 279 raft_consensus.cc:571] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [term 1 LEADER]: Becoming Leader. State: Replica: 476aed1601be4605b69e0622e3eeb7fc, State: Running, Role: LEADER I0507 20:05:14.795732 279 consensus_queue.cc:204] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476aed1601be4605b69e0622e3eeb7fc" member_type: VOTER } I0507 20:05:14.798727 276 sys_catalog.cc:335] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "476aed1601be4605b69e0622e3eeb7fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476aed1601be4605b69e0622e3eeb7fc" member_type: VOTER } } I0507 20:05:14.799053 276 sys_catalog.cc:338] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [sys.catalog]: This master's current role is: LEADER I0507 20:05:14.799042 280 sys_catalog.cc:335] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "476aed1601be4605b69e0622e3eeb7fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476aed1601be4605b69e0622e3eeb7fc" member_type: VOTER } } I0507 20:05:14.800915 280 sys_catalog.cc:338] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [sys.catalog]: This master's current role is: LEADER I0507 20:05:14.799268 281 sys_catalog.cc:335] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 476aed1601be4605b69e0622e3eeb7fc. Latest consensus state: current_term: 1 leader_uuid: "476aed1601be4605b69e0622e3eeb7fc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "476aed1601be4605b69e0622e3eeb7fc" member_type: VOTER } } I0507 20:05:14.801246 281 sys_catalog.cc:338] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [sys.catalog]: This master's current role is: LEADER I0507 20:05:14.802125 276 sys_catalog.cc:422] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc [sys.catalog]: configured and running, proceeding with master startup. I0507 20:05:14.802320 283 catalog_manager.cc:990] Loading table and tablet metadata into memory... I0507 20:05:14.802865 217 master_main.cc:79] Master server successfully started. I0507 20:05:14.806584 283 catalog_manager.cc:1001] Initializing Kudu internal certificate authority... I0507 20:05:14.857975 283 catalog_manager.cc:888] Generated new certificate authority record I0507 20:05:14.860021 283 catalog_manager.cc:1010] Loading token signing keys... I0507 20:05:14.881369 283 catalog_manager.cc:3992] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Generated new TSK 0 WARNING: Logging before InitGoogleLogging() is written to STDERR W0507 20:05:14.946125 277 flags.cc:412] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0507 20:05:14.946399 277 flags.cc:412] Enabled unsafe flag: --never_fsync=true W0507 20:05:14.948706 277 flags.cc:412] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000 W0507 20:05:14.948778 277 flags.cc:412] Enabled experimental flag: --ipki_server_key_size=1024 W0507 20:05:14.948915 277 flags.cc:412] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.52.193 I0507 20:05:14.954172 277 tablet_server_main.cc:72] Tablet server non-default flags: --leader_failure_exp_backoff_max_delta_ms=1000 --raft_heartbeat_interval_ms=25 --fs_data_dirs=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data --fs_wal_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.0.52.193:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data/info.pb --webserver_interface=127.0.52.193 --webserver_port=0 --heartbeat_interval_ms=25 --tserver_master_addrs=127.0.52.254:52118 --never_fsync=true --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.0.52.193 --log_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.8.0-cdh5.16.0-SNAPSHOT revision 4253c68ab2140f5294ab236d1c0fc445ad80f766 build type FASTDEBUG built by jenkins at 07 May 2018 12:28:50 PST on quasar-snqeym-1.vpc.cloudera.com build id 50748 ASAN enabled I0507 20:05:14.960497 277 tablet_server_main.cc:79] Initializing tablet server... I0507 20:05:14.962939 277 system_ntp.cc:143] NTP initialized. Skew: 500ppm Current error: 1158388us I0507 20:05:14.963801 277 fs_manager.cc:260] Metadata directory not provided I0507 20:05:14.963873 277 fs_manager.cc:266] Using write-ahead log directory (fs_wal_dir) as metadata directory I0507 20:05:14.964138 277 server_base.cc:433] Could not load existing FS layout: Not found: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data/instance: No such file or directory (error 2) I0507 20:05:14.964231 277 server_base.cc:434] Attempting to create new FS layout instead I0507 20:05:14.968294 277 fs_manager.cc:595] Generated new instance metadata in path /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data/instance: uuid: "d1bf0f72a88f4483a1906bc8246849dd" format_stamp: "Formatted at 2018-05-07 20:05:14 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:14.968969 277 fs_manager.cc:595] Generated new instance metadata in path /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/wal/instance: uuid: "d1bf0f72a88f4483a1906bc8246849dd" format_stamp: "Formatted at 2018-05-07 20:05:14 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:14.977416 277 fs_manager.cc:495] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s I0507 20:05:14.978546 277 env_posix.cc:1643] Not raising this process' open files per process limit of 65536; it is already as high as it can go I0507 20:05:14.979118 277 file_cache.cc:470] Constructed file cache lbm with capacity 26214 I0507 20:05:14.982980 277 fs_manager.cc:417] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s I0507 20:05:14.983055 277 fs_manager.cc:428] Opened local filesystem: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data,/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/wal uuid: "d1bf0f72a88f4483a1906bc8246849dd" format_stamp: "Formatted at 2018-05-07 20:05:14 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:14.983240 277 fs_report.cc:347] Block manager report -------------------- 1 data directories: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0507 20:05:15.018365 277 env_posix.cc:1643] Not raising this process' running threads per effective uid limit of 60284; it is already as high as it can go I0507 20:05:15.020113 277 ts_tablet_manager.cc:356] Loaded tablet metadata (0 live tablets) I0507 20:05:15.020365 277 tablet_server_main.cc:84] Starting tablet server... I0507 20:05:15.052618 277 rpc_server.cc:205] RPC server started. Bound to: 127.0.52.193:46057 I0507 20:05:15.053160 277 webserver.cc:173] Starting webserver on 127.0.52.193:0 I0507 20:05:15.053206 277 webserver.cc:184] Document root disabled I0507 20:05:15.053949 277 webserver.cc:311] Webserver started. Bound to: http://127.0.52.193:39774/ I0507 20:05:15.054728 277 server_base.cc:610] Dumped server information to /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-0/data/info.pb I0507 20:05:15.061095 211 external_mini_cluster.cc:848] Started /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver as pid 277 I0507 20:05:15.061924 211 external_mini_cluster.cc:786] Running /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/wal --fs_data_dirs=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data --block_manager=log --rpc_bind_addresses=127.0.52.194:0 --local_ip_for_outbound_sockets=127.0.52.194 --webserver_interface=127.0.52.194 --webserver_port=0 --tserver_master_addrs=127.0.52.254:52118 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/logs --server_dump_info_path=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --raft_heartbeat_interval_ms=25 --leader_failure_exp_backoff_max_delta_ms=1000 --heartbeat_interval_ms=25 with env {} I0507 20:05:15.064160 277 tablet_server_main.cc:87] Tablet server successfully started. I0507 20:05:15.079107 406 heartbeater.cc:346] Connected to a master server at 127.0.52.254:52118 I0507 20:05:15.079329 406 heartbeater.cc:426] Registering TS with master... I0507 20:05:15.079923 406 heartbeater.cc:475] Master 127.0.52.254:52118 requested a full tablet report, sending... I0507 20:05:15.081797 243 ts_manager.cc:81] Registered new tserver with Master: d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057) I0507 20:05:15.083586 243 master_service.cc:249] Signed X509 certificate for tserver {username='slave'} at 127.0.52.193:59348 I0507 20:05:15.111245 406 heartbeater.cc:467] Master 127.0.52.254:52118 was elected leader, sending a full tablet report... WARNING: Logging before InitGoogleLogging() is written to STDERR W0507 20:05:15.271286 410 flags.cc:412] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0507 20:05:15.271536 410 flags.cc:412] Enabled unsafe flag: --never_fsync=true W0507 20:05:15.273543 410 flags.cc:412] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000 W0507 20:05:15.273597 410 flags.cc:412] Enabled experimental flag: --ipki_server_key_size=1024 W0507 20:05:15.273689 410 flags.cc:412] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.52.194 I0507 20:05:15.278712 410 tablet_server_main.cc:72] Tablet server non-default flags: --leader_failure_exp_backoff_max_delta_ms=1000 --raft_heartbeat_interval_ms=25 --fs_data_dirs=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data --fs_wal_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.0.52.194:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data/info.pb --webserver_interface=127.0.52.194 --webserver_port=0 --heartbeat_interval_ms=25 --tserver_master_addrs=127.0.52.254:52118 --never_fsync=true --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.0.52.194 --log_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.8.0-cdh5.16.0-SNAPSHOT revision 4253c68ab2140f5294ab236d1c0fc445ad80f766 build type FASTDEBUG built by jenkins at 07 May 2018 12:28:50 PST on quasar-snqeym-1.vpc.cloudera.com build id 50748 ASAN enabled I0507 20:05:15.284699 410 tablet_server_main.cc:79] Initializing tablet server... I0507 20:05:15.286186 410 system_ntp.cc:143] NTP initialized. Skew: 500ppm Current error: 1158888us I0507 20:05:15.286942 410 fs_manager.cc:260] Metadata directory not provided I0507 20:05:15.286998 410 fs_manager.cc:266] Using write-ahead log directory (fs_wal_dir) as metadata directory I0507 20:05:15.287214 410 server_base.cc:433] Could not load existing FS layout: Not found: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data/instance: No such file or directory (error 2) I0507 20:05:15.287289 410 server_base.cc:434] Attempting to create new FS layout instead I0507 20:05:15.290916 410 fs_manager.cc:595] Generated new instance metadata in path /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data/instance: uuid: "5014824b94b64fd49da2c1a645c078aa" format_stamp: "Formatted at 2018-05-07 20:05:15 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:15.291465 410 fs_manager.cc:595] Generated new instance metadata in path /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/wal/instance: uuid: "5014824b94b64fd49da2c1a645c078aa" format_stamp: "Formatted at 2018-05-07 20:05:15 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:15.298626 410 fs_manager.cc:495] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s I0507 20:05:15.299672 410 env_posix.cc:1643] Not raising this process' open files per process limit of 65536; it is already as high as it can go I0507 20:05:15.300163 410 file_cache.cc:470] Constructed file cache lbm with capacity 26214 I0507 20:05:15.303961 410 fs_manager.cc:417] Time spent opening block manager: real 0.002s user 0.000s sys 0.004s I0507 20:05:15.304021 410 fs_manager.cc:428] Opened local filesystem: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data,/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/wal uuid: "5014824b94b64fd49da2c1a645c078aa" format_stamp: "Formatted at 2018-05-07 20:05:15 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:15.304175 410 fs_report.cc:347] Block manager report -------------------- 1 data directories: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0507 20:05:15.326686 410 env_posix.cc:1643] Not raising this process' running threads per effective uid limit of 60284; it is already as high as it can go I0507 20:05:15.328233 410 ts_tablet_manager.cc:356] Loaded tablet metadata (0 live tablets) I0507 20:05:15.328442 410 tablet_server_main.cc:84] Starting tablet server... I0507 20:05:15.351519 410 rpc_server.cc:205] RPC server started. Bound to: 127.0.52.194:43942 I0507 20:05:15.352789 410 webserver.cc:173] Starting webserver on 127.0.52.194:0 I0507 20:05:15.352833 410 webserver.cc:184] Document root disabled I0507 20:05:15.353536 410 webserver.cc:311] Webserver started. Bound to: http://127.0.52.194:50872/ I0507 20:05:15.354228 410 server_base.cc:610] Dumped server information to /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-1/data/info.pb I0507 20:05:15.355111 410 tablet_server_main.cc:87] Tablet server successfully started. I0507 20:05:15.363253 211 external_mini_cluster.cc:848] Started /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver as pid 410 I0507 20:05:15.364084 211 external_mini_cluster.cc:786] Running /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver --fs_wal_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/wal --fs_data_dirs=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data --block_manager=log --rpc_bind_addresses=127.0.52.195:0 --local_ip_for_outbound_sockets=127.0.52.195 --webserver_interface=127.0.52.195 --webserver_port=0 --tserver_master_addrs=127.0.52.254:52118 --never_fsync --ipki_server_key_size=1024 --enable_minidumps=false --redact=none --metrics_log_interval_ms=1000 --logtostderr --logbuflevel=-1 --log_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/logs --server_dump_info_path=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data/info.pb --server_dump_info_format=pb --rpc_server_allow_ephemeral_ports --unlock_experimental_flags --unlock_unsafe_flags --raft_heartbeat_interval_ms=25 --leader_failure_exp_backoff_max_delta_ms=1000 --heartbeat_interval_ms=25 with env {} I0507 20:05:15.368650 529 heartbeater.cc:346] Connected to a master server at 127.0.52.254:52118 I0507 20:05:15.368860 529 heartbeater.cc:426] Registering TS with master... I0507 20:05:15.369392 529 heartbeater.cc:475] Master 127.0.52.254:52118 requested a full tablet report, sending... I0507 20:05:15.370645 243 ts_manager.cc:81] Registered new tserver with Master: 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942) I0507 20:05:15.371932 243 master_service.cc:249] Signed X509 certificate for tserver {username='slave'} at 127.0.52.194:58584 I0507 20:05:15.399281 529 heartbeater.cc:467] Master 127.0.52.254:52118 was elected leader, sending a full tablet report... WARNING: Logging before InitGoogleLogging() is written to STDERR W0507 20:05:15.526067 532 flags.cc:412] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true W0507 20:05:15.526286 532 flags.cc:412] Enabled unsafe flag: --never_fsync=true W0507 20:05:15.528301 532 flags.cc:412] Enabled experimental flag: --leader_failure_exp_backoff_max_delta_ms=1000 W0507 20:05:15.528352 532 flags.cc:412] Enabled experimental flag: --ipki_server_key_size=1024 W0507 20:05:15.528443 532 flags.cc:412] Enabled experimental flag: --local_ip_for_outbound_sockets=127.0.52.195 I0507 20:05:15.532126 532 tablet_server_main.cc:72] Tablet server non-default flags: --leader_failure_exp_backoff_max_delta_ms=1000 --raft_heartbeat_interval_ms=25 --fs_data_dirs=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data --fs_wal_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/wal --ipki_server_key_size=1024 --rpc_bind_addresses=127.0.52.195:0 --rpc_server_allow_ephemeral_ports=true --metrics_log_interval_ms=1000 --server_dump_info_format=pb --server_dump_info_path=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data/info.pb --webserver_interface=127.0.52.195 --webserver_port=0 --heartbeat_interval_ms=25 --tserver_master_addrs=127.0.52.254:52118 --never_fsync=true --redact=none --unlock_experimental_flags=true --unlock_unsafe_flags=true --enable_minidumps=false --local_ip_for_outbound_sockets=127.0.52.195 --log_dir=/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/logs --logbuflevel=-1 --logtostderr=true Tablet server version: kudu 1.8.0-cdh5.16.0-SNAPSHOT revision 4253c68ab2140f5294ab236d1c0fc445ad80f766 build type FASTDEBUG built by jenkins at 07 May 2018 12:28:50 PST on quasar-snqeym-1.vpc.cloudera.com build id 50748 ASAN enabled I0507 20:05:15.537900 532 tablet_server_main.cc:79] Initializing tablet server... I0507 20:05:15.540156 532 system_ntp.cc:143] NTP initialized. Skew: 500ppm Current error: 1158888us I0507 20:05:15.540889 532 fs_manager.cc:260] Metadata directory not provided I0507 20:05:15.540949 532 fs_manager.cc:266] Using write-ahead log directory (fs_wal_dir) as metadata directory I0507 20:05:15.541167 532 server_base.cc:433] Could not load existing FS layout: Not found: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data/instance: No such file or directory (error 2) I0507 20:05:15.541240 532 server_base.cc:434] Attempting to create new FS layout instead I0507 20:05:15.544921 532 fs_manager.cc:595] Generated new instance metadata in path /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data/instance: uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" format_stamp: "Formatted at 2018-05-07 20:05:15 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:15.545452 532 fs_manager.cc:595] Generated new instance metadata in path /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/wal/instance: uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" format_stamp: "Formatted at 2018-05-07 20:05:15 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:15.552822 532 fs_manager.cc:495] Time spent creating directory manager: real 0.007s user 0.008s sys 0.004s I0507 20:05:15.553804 532 env_posix.cc:1643] Not raising this process' open files per process limit of 65536; it is already as high as it can go I0507 20:05:15.554286 532 file_cache.cc:470] Constructed file cache lbm with capacity 26214 I0507 20:05:15.557632 532 fs_manager.cc:417] Time spent opening block manager: real 0.002s user 0.000s sys 0.004s I0507 20:05:15.557683 532 fs_manager.cc:428] Opened local filesystem: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data,/tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/wal uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" format_stamp: "Formatted at 2018-05-07 20:05:15 on dist-test-slave-dist-test-slave-8njq" I0507 20:05:15.557831 532 fs_report.cc:347] Block manager report -------------------- 1 data directories: /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0507 20:05:15.580082 532 env_posix.cc:1643] Not raising this process' running threads per effective uid limit of 60284; it is already as high as it can go I0507 20:05:15.581683 532 ts_tablet_manager.cc:356] Loaded tablet metadata (0 live tablets) I0507 20:05:15.581898 532 tablet_server_main.cc:84] Starting tablet server... I0507 20:05:15.607074 532 rpc_server.cc:205] RPC server started. Bound to: 127.0.52.195:38453 I0507 20:05:15.607560 532 webserver.cc:173] Starting webserver on 127.0.52.195:0 I0507 20:05:15.607597 532 webserver.cc:184] Document root disabled I0507 20:05:15.608327 532 webserver.cc:311] Webserver started. Bound to: http://127.0.52.195:41566/ I0507 20:05:15.609058 532 server_base.cc:610] Dumped server information to /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211/minicluster-data/ts-2/data/info.pb I0507 20:05:15.609875 532 tablet_server_main.cc:87] Tablet server successfully started. I0507 20:05:15.614354 211 external_mini_cluster.cc:848] Started /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver as pid 532 I0507 20:05:15.628906 651 heartbeater.cc:346] Connected to a master server at 127.0.52.254:52118 I0507 20:05:15.629129 651 heartbeater.cc:426] Registering TS with master... I0507 20:05:15.629762 651 heartbeater.cc:475] Master 127.0.52.254:52118 requested a full tablet report, sending... I0507 20:05:15.631110 242 ts_manager.cc:81] Registered new tserver with Master: b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) I0507 20:05:15.632138 242 master_service.cc:249] Signed X509 certificate for tserver {username='slave'} at 127.0.52.195:40189 I0507 20:05:15.638284 211 external_mini_cluster.cc:482] 3 TS(s) registered with all masters I0507 20:05:15.659437 651 heartbeater.cc:467] Master 127.0.52.254:52118 was elected leader, sending a full tablet report... I0507 20:05:15.659670 242 catalog_manager.cc:1316] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44037: name: "kudu1580" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "val" type: INT32 is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 3 split_rows_range_bounds { } partition_schema { hash_bucket_schemas { columns { name: "key" } num_buckets: 3 seed: 0 } } W0507 20:05:15.665654 242 catalog_manager.cc:1472] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu1580 in case of a replica failure: 4 tablet servers are needed, 3 are alive. Consider bringing up additional tablet server(s) or running both the masters and all tablet servers with --raft_prepare_replacement_before_eviction=false flag (not recommended). I0507 20:05:15.694381 588 tablet_service.cc:760] Processing CreateTablet for tablet a0376a5e3b9f495aa00d366b35594a27 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.694198 466 tablet_service.cc:760] Processing CreateTablet for tablet a0376a5e3b9f495aa00d366b35594a27 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.695341 588 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet a0376a5e3b9f495aa00d366b35594a27. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.695303 465 tablet_service.cc:760] Processing CreateTablet for tablet d80f4fae4b9f4d238f7f626507290390 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.695817 464 tablet_service.cc:760] Processing CreateTablet for tablet 7161aaf1a71d455883ef478337dfaf19 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.696483 465 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet d80f4fae4b9f4d238f7f626507290390. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.703197 464 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet 7161aaf1a71d455883ef478337dfaf19. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.703713 587 tablet_service.cc:760] Processing CreateTablet for tablet d80f4fae4b9f4d238f7f626507290390 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.704110 587 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet d80f4fae4b9f4d238f7f626507290390. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.705356 342 tablet_service.cc:760] Processing CreateTablet for tablet d80f4fae4b9f4d238f7f626507290390 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 1, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.706423 342 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet d80f4fae4b9f4d238f7f626507290390. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.707290 341 tablet_service.cc:760] Processing CreateTablet for tablet 7161aaf1a71d455883ef478337dfaf19 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.708920 341 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet 7161aaf1a71d455883ef478337dfaf19. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.710084 466 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet a0376a5e3b9f495aa00d366b35594a27. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.711434 343 tablet_service.cc:760] Processing CreateTablet for tablet a0376a5e3b9f495aa00d366b35594a27 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 0, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.711612 586 tablet_service.cc:760] Processing CreateTablet for tablet 7161aaf1a71d455883ef478337dfaf19 (table=kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]), partition=HASH (key) PARTITION 2, RANGE (key) PARTITION UNBOUNDED I0507 20:05:15.711748 343 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet a0376a5e3b9f495aa00d366b35594a27. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.711971 586 data_dirs.cc:934] Could only allocate 1 dirs of requested 3 for tablet 7161aaf1a71d455883ef478337dfaf19. 1 dirs total, 0 dirs full, 0 dirs failed I0507 20:05:15.717669 586 ts_tablet_manager.cc:1173] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.717872 466 ts_tablet_manager.cc:1173] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.718639 587 ts_tablet_manager.cc:1173] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.721312 465 ts_tablet_manager.cc:1173] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.723114 464 ts_tablet_manager.cc:1173] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.724510 671 ts_tablet_manager.cc:1028] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Bootstrapping tablet I0507 20:05:15.724851 671 tablet_bootstrap.cc:436] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Bootstrap starting. I0507 20:05:15.727761 671 tablet_bootstrap.cc:587] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: No blocks or log segments found. Creating new log. I0507 20:05:15.730360 671 log.cc:520] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Log is configured to *not* fsync() on all Append() calls I0507 20:05:15.735154 670 ts_tablet_manager.cc:1028] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Bootstrapping tablet I0507 20:05:15.735585 670 tablet_bootstrap.cc:436] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Bootstrap starting. I0507 20:05:15.743778 671 tablet_bootstrap.cc:436] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: No bootstrap required, opened a new log I0507 20:05:15.744091 671 ts_tablet_manager.cc:1045] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Time spent bootstrapping tablet: real 0.019s user 0.008s sys 0.008s I0507 20:05:15.744670 588 ts_tablet_manager.cc:1173] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.745487 343 ts_tablet_manager.cc:1173] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.746054 341 ts_tablet_manager.cc:1173] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.746758 673 ts_tablet_manager.cc:1028] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Bootstrapping tablet I0507 20:05:15.747004 342 ts_tablet_manager.cc:1173] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Registered tablet (data state: TABLET_DATA_READY) I0507 20:05:15.747117 673 tablet_bootstrap.cc:436] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Bootstrap starting. I0507 20:05:15.747455 670 tablet_bootstrap.cc:587] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: No blocks or log segments found. Creating new log. I0507 20:05:15.750026 670 log.cc:520] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Log is configured to *not* fsync() on all Append() calls I0507 20:05:15.752017 671 raft_consensus.cc:303] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:15.752415 671 raft_consensus.cc:329] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.752501 671 raft_consensus.cc:604] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Initialized, Role: FOLLOWER I0507 20:05:15.753134 671 consensus_queue.cc:227] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:15.753798 673 tablet_bootstrap.cc:587] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: No blocks or log segments found. Creating new log. I0507 20:05:15.754848 673 log.cc:520] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Log is configured to *not* fsync() on all Append() calls I0507 20:05:15.755084 670 tablet_bootstrap.cc:436] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: No bootstrap required, opened a new log I0507 20:05:15.755373 670 ts_tablet_manager.cc:1045] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Time spent bootstrapping tablet: real 0.020s user 0.000s sys 0.008s I0507 20:05:15.759807 671 ts_tablet_manager.cc:1073] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Time spent starting tablet: real 0.016s user 0.008s sys 0.004s I0507 20:05:15.760043 671 ts_tablet_manager.cc:1028] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Bootstrapping tablet I0507 20:05:15.760200 671 tablet_bootstrap.cc:436] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Bootstrap starting. I0507 20:05:15.761389 671 tablet_bootstrap.cc:587] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: No blocks or log segments found. Creating new log. I0507 20:05:15.763478 671 tablet_bootstrap.cc:436] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: No bootstrap required, opened a new log I0507 20:05:15.763595 671 ts_tablet_manager.cc:1045] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.004s I0507 20:05:15.763741 673 tablet_bootstrap.cc:436] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: No bootstrap required, opened a new log I0507 20:05:15.764031 673 ts_tablet_manager.cc:1045] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.004s I0507 20:05:15.764484 671 raft_consensus.cc:303] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.764679 671 raft_consensus.cc:329] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.764349 670 raft_consensus.cc:303] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.766294 670 raft_consensus.cc:329] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.766369 670 raft_consensus.cc:604] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e887ddd8a14cf3b2c24a0aa14be025, State: Initialized, Role: FOLLOWER I0507 20:05:15.766571 675 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:15.766695 675 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.103s (starting election) I0507 20:05:15.766804 675 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:15.767050 670 consensus_queue.cc:227] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.767879 675 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.773232 673 raft_consensus.cc:303] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:15.773667 673 raft_consensus.cc:329] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.773732 673 raft_consensus.cc:604] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Initialized, Role: FOLLOWER I0507 20:05:15.774286 673 consensus_queue.cc:227] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:15.775838 675 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.776450 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:15.778172 671 raft_consensus.cc:604] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Initialized, Role: FOLLOWER I0507 20:05:15.778368 671 consensus_queue.cc:227] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.779320 671 ts_tablet_manager.cc:1073] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Time spent starting tablet: real 0.015s user 0.004s sys 0.000s I0507 20:05:15.779449 671 ts_tablet_manager.cc:1028] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Bootstrapping tablet I0507 20:05:15.779626 671 tablet_bootstrap.cc:436] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Bootstrap starting. I0507 20:05:15.779708 670 ts_tablet_manager.cc:1073] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Time spent starting tablet: real 0.024s user 0.016s sys 0.008s I0507 20:05:15.779950 670 ts_tablet_manager.cc:1028] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Bootstrapping tablet I0507 20:05:15.780119 670 tablet_bootstrap.cc:436] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Bootstrap starting. W0507 20:05:15.780498 420 leader_election.cc:292] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Illegal state: must be running to vote when last-logged opid is not known I0507 20:05:15.781491 670 tablet_bootstrap.cc:587] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: No blocks or log segments found. Creating new log. I0507 20:05:15.782361 673 ts_tablet_manager.cc:1073] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Time spent starting tablet: real 0.018s user 0.008s sys 0.004s I0507 20:05:15.782562 673 ts_tablet_manager.cc:1028] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Bootstrapping tablet I0507 20:05:15.782678 673 tablet_bootstrap.cc:436] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Bootstrap starting. I0507 20:05:15.783171 670 tablet_bootstrap.cc:436] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: No bootstrap required, opened a new log I0507 20:05:15.783287 670 ts_tablet_manager.cc:1045] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Time spent bootstrapping tablet: real 0.003s user 0.004s sys 0.000s I0507 20:05:15.785383 671 tablet_bootstrap.cc:587] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: No blocks or log segments found. Creating new log. I0507 20:05:15.785552 673 tablet_bootstrap.cc:587] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: No blocks or log segments found. Creating new log. I0507 20:05:15.785454 670 raft_consensus.cc:303] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.785636 670 raft_consensus.cc:329] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.785682 670 raft_consensus.cc:604] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e887ddd8a14cf3b2c24a0aa14be025, State: Initialized, Role: FOLLOWER I0507 20:05:15.785900 670 consensus_queue.cc:227] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.786777 670 ts_tablet_manager.cc:1073] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s I0507 20:05:15.786937 670 ts_tablet_manager.cc:1028] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Bootstrapping tablet I0507 20:05:15.787071 670 tablet_bootstrap.cc:436] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Bootstrap starting. I0507 20:05:15.788261 678 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:15.788405 678 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.080s (starting election) I0507 20:05:15.788520 678 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.789479 678 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:15.791297 678 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.792703 363 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:15.793076 363 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.098s (vote granted) I0507 20:05:15.793203 363 raft_consensus.cc:2219] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 0. I0507 20:05:15.794042 420 leader_election.cc:362] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Vote granted by peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.794134 420 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I0507 20:05:15.794792 675 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.088s (election complete) I0507 20:05:15.794909 675 raft_consensus.cc:2511] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Leader pre-election won for term 1 I0507 20:05:15.794996 675 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0507 20:05:15.795064 675 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.081s (starting election) I0507 20:05:15.795122 675 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:15.797791 675 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:15.798290 675 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Requesting vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.798501 675 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Requesting vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.800683 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" I0507 20:05:15.800868 363 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true W0507 20:05:15.801410 420 leader_election.cc:292] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Illegal state: must be running to vote when last-logged opid is not known I0507 20:05:15.802186 363 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" W0507 20:05:15.802294 542 leader_election.cc:292] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Illegal state: must be running to vote when last-logged opid is not known I0507 20:05:15.802335 363 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:15.802476 678 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:15.802484 363 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.090s (vote granted) I0507 20:05:15.802548 678 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.106s (starting election) I0507 20:05:15.802634 678 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.802944 670 tablet_bootstrap.cc:587] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: No blocks or log segments found. Creating new log. I0507 20:05:15.803088 678 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:15.803542 678 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.804358 675 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:15.804443 675 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.106s (starting election) I0507 20:05:15.804527 675 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.804920 675 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.805083 486 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:15.805408 675 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.805809 485 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:15.806059 485 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.112s (vote granted) I0507 20:05:15.806063 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:15.806146 485 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 0. I0507 20:05:15.806215 608 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.092s (vote granted) I0507 20:05:15.806282 608 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 0. I0507 20:05:15.806793 420 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:15.806792 541 leader_election.cc:292] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Illegal state: must be running to vote when last-logged opid is not known I0507 20:05:15.806849 420 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I0507 20:05:15.806865 541 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I0507 20:05:15.807493 675 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.076s (election complete) I0507 20:05:15.807526 541 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Vote granted by peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:15.807572 541 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I0507 20:05:15.807565 675 raft_consensus.cc:2511] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Leader pre-election won for term 1 I0507 20:05:15.807617 675 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0507 20:05:15.807659 675 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.103s (starting election) I0507 20:05:15.807714 675 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:15.807834 678 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.079s (election complete) I0507 20:05:15.807883 678 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I0507 20:05:15.807997 678 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.101s (election complete) I0507 20:05:15.808044 678 raft_consensus.cc:2511] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Leader pre-election won for term 1 I0507 20:05:15.808089 678 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0507 20:05:15.808123 678 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.105s (starting election) I0507 20:05:15.808168 678 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:15.810690 675 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.811094 675 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Requesting vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.811231 675 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Requesting vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.812258 671 tablet_bootstrap.cc:436] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: No bootstrap required, opened a new log I0507 20:05:15.812392 671 ts_tablet_manager.cc:1045] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Time spent bootstrapping tablet: real 0.033s user 0.004s sys 0.012s I0507 20:05:15.812542 678 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.812873 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" I0507 20:05:15.812993 678 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 election: Requesting vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:15.813100 608 raft_consensus.cc:2149] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa in current term 1: Already voted for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in this term. I0507 20:05:15.813139 678 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 election: Requesting vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.813094 671 raft_consensus.cc:303] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.813263 671 raft_consensus.cc:329] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.813313 671 raft_consensus.cc:604] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Initialized, Role: FOLLOWER I0507 20:05:15.813701 671 consensus_queue.cc:227] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.813923 675 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:15.813982 675 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.081s (starting election) I0507 20:05:15.814054 675 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.814378 675 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.814507 675 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.814997 671 ts_tablet_manager.cc:1073] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s I0507 20:05:15.816071 420 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa in current term 1: Already voted for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in this term. I0507 20:05:15.816656 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:15.816833 608 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.130s (vote granted) I0507 20:05:15.816895 608 raft_consensus.cc:2219] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 0. I0507 20:05:15.817833 420 leader_election.cc:362] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.817903 420 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I0507 20:05:15.818107 683 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.099s (election complete) I0507 20:05:15.818214 683 raft_consensus.cc:2511] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Leader pre-election won for term 1 I0507 20:05:15.818284 683 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0507 20:05:15.818348 683 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.091s (starting election) I0507 20:05:15.818400 683 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 0 FOLLOWER]: Advancing to term 1 W0507 20:05:15.818536 407 tablet.cc:1678] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Can't schedule compaction. Clean time has not been advanced past its initial value. I0507 20:05:15.820546 683 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.821357 683 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Requesting vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.821776 683 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Requesting vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.804289 362 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:15.822402 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" I0507 20:05:15.822563 608 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:15.822638 608 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.098s (vote granted) I0507 20:05:15.822127 360 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" I0507 20:05:15.822329 361 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:15.821538 358 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:15.823348 359 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" I0507 20:05:15.824124 362 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" I0507 20:05:15.824867 608 raft_consensus.cc:2219] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 1. W0507 20:05:15.825366 542 leader_election.cc:292] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Illegal state: must be running to vote when last-logged opid is not known W0507 20:05:15.825681 542 leader_election.cc:292] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Illegal state: must be running to vote when last-logged opid is not known I0507 20:05:15.825821 420 leader_election.cc:362] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.825903 420 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. W0507 20:05:15.826211 420 leader_election.cc:292] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Illegal state: must be running to vote when last-logged opid is not known I0507 20:05:15.826516 673 tablet_bootstrap.cc:436] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: No bootstrap required, opened a new log W0507 20:05:15.826609 420 leader_election.cc:292] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Illegal state: must be running to vote when last-logged opid is not known I0507 20:05:15.826650 673 ts_tablet_manager.cc:1045] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Time spent bootstrapping tablet: real 0.044s user 0.004s sys 0.004s I0507 20:05:15.826818 420 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. I0507 20:05:15.827028 675 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.093s (election complete) I0507 20:05:15.827100 675 raft_consensus.cc:2511] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Leader election won for term 1 I0507 20:05:15.827507 673 raft_consensus.cc:303] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.827708 673 raft_consensus.cc:329] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.827754 673 raft_consensus.cc:604] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Initialized, Role: FOLLOWER I0507 20:05:15.827956 673 consensus_queue.cc:227] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:15.829473 420 leader_election.cc:292] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Illegal state: must be running to vote when last-logged opid is not known W0507 20:05:15.830251 420 leader_election.cc:292] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Illegal state: must be running to vote when last-logged opid is not known I0507 20:05:15.832355 679 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:15.832459 679 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.092s (starting election) I0507 20:05:15.832546 679 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.833535 679 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:15.833712 363 raft_consensus.cc:2219] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 1. I0507 20:05:15.833900 679 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.834503 420 leader_election.cc:362] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Vote granted by peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.834604 420 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I0507 20:05:15.838786 683 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.096s (election complete) I0507 20:05:15.839174 683 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority I0507 20:05:15.839821 673 ts_tablet_manager.cc:1073] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Time spent starting tablet: real 0.013s user 0.004s sys 0.000s I0507 20:05:15.839939 683 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.095s (election complete) I0507 20:05:15.839960 673 ts_tablet_manager.cc:1028] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Bootstrapping tablet I0507 20:05:15.840119 673 tablet_bootstrap.cc:436] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Bootstrap starting. I0507 20:05:15.840255 683 raft_consensus.cc:2511] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Leader election won for term 1 I0507 20:05:15.840488 683 raft_consensus.cc:571] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Becoming Leader. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Running, Role: LEADER I0507 20:05:15.841167 683 consensus_queue.cc:204] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:15.841681 673 tablet_bootstrap.cc:587] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: No blocks or log segments found. Creating new log. I0507 20:05:15.846860 485 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" I0507 20:05:15.847172 485 raft_consensus.cc:2149] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 in current term 1: Already voted for candidate 5014824b94b64fd49da2c1a645c078aa in this term. I0507 20:05:15.847419 673 tablet_bootstrap.cc:436] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: No bootstrap required, opened a new log I0507 20:05:15.847555 673 ts_tablet_manager.cc:1045] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.004s I0507 20:05:15.847978 541 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 election: Vote denied by peer 5014824b94b64fd49da2c1a645c078aa. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 in current term 1: Already voted for candidate 5014824b94b64fd49da2c1a645c078aa in this term. I0507 20:05:15.848141 541 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. I0507 20:05:15.848484 684 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.094s (election complete) I0507 20:05:15.848590 684 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority I0507 20:05:15.848636 673 raft_consensus.cc:303] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.848815 673 raft_consensus.cc:329] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.848860 673 raft_consensus.cc:604] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Initialized, Role: FOLLOWER I0507 20:05:15.848927 675 raft_consensus.cc:571] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Becoming Leader. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Running, Role: LEADER I0507 20:05:15.849040 673 consensus_queue.cc:227] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.849290 675 consensus_queue.cc:204] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.851673 673 ts_tablet_manager.cc:1073] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Time spent starting tablet: real 0.004s user 0.000s sys 0.000s W0507 20:05:15.856155 530 tablet.cc:1678] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Can't schedule compaction. Clean time has not been advanced past its initial value. I0507 20:05:15.857195 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:15.857385 670 tablet_bootstrap.cc:436] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: No bootstrap required, opened a new log I0507 20:05:15.857467 608 raft_consensus.cc:2149] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd in current term 1: Already voted for candidate 5014824b94b64fd49da2c1a645c078aa in this term. I0507 20:05:15.858422 670 ts_tablet_manager.cc:1045] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Time spent bootstrapping tablet: real 0.071s user 0.000s sys 0.008s I0507 20:05:15.859011 297 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd in current term 1: Already voted for candidate 5014824b94b64fd49da2c1a645c078aa in this term. I0507 20:05:15.859016 485 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:15.859169 670 raft_consensus.cc:303] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:15.859359 670 raft_consensus.cc:329] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0507 20:05:15.859407 670 raft_consensus.cc:604] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5e887ddd8a14cf3b2c24a0aa14be025, State: Initialized, Role: FOLLOWER I0507 20:05:15.859583 670 consensus_queue.cc:227] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } W0507 20:05:15.859928 652 tablet.cc:1678] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Can't schedule compaction. Clean time has not been advanced past its initial value. I0507 20:05:15.861130 670 ts_tablet_manager.cc:1073] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s I0507 20:05:15.862498 485 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 1 because replica is either leader or believes a valid leader to be alive. I0507 20:05:15.863190 679 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:15.863214 296 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Vote denied by peer 5014824b94b64fd49da2c1a645c078aa. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 1 because replica is either leader or believes a valid leader to be alive. I0507 20:05:15.863287 296 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I0507 20:05:15.863281 679 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.094s (starting election) I0507 20:05:15.863354 679 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.863765 679 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:15.863894 679 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.863459 242 catalog_manager.cc:3716] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa reported cstate change: term changed from 0 to 1, leader changed from to 5014824b94b64fd49da2c1a645c078aa (127.0.52.194). New cstate: current_term: 1 leader_uuid: "5014824b94b64fd49da2c1a645c078aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: UNKNOWN } } } I0507 20:05:15.864074 679 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.103s (election complete) I0507 20:05:15.864135 679 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:15.866379 679 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I0507 20:05:15.867118 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:15.867326 608 raft_consensus.cc:2149] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd in current term 1: Already voted for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in this term. I0507 20:05:15.867761 607 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:15.867799 297 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd in current term 1: Already voted for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in this term. I0507 20:05:15.867934 485 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:15.868136 485 raft_consensus.cc:2149] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd in current term 1: Already voted for candidate 5014824b94b64fd49da2c1a645c078aa in this term. I0507 20:05:15.868629 296 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Vote denied by peer 5014824b94b64fd49da2c1a645c078aa. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd in current term 1: Already voted for candidate 5014824b94b64fd49da2c1a645c078aa in this term. I0507 20:05:15.868685 296 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I0507 20:05:15.868978 679 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.110s (election complete) I0507 20:05:15.869055 679 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 0 FOLLOWER]: Advancing to term 1 I0507 20:05:15.869298 242 catalog_manager.cc:3716] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa reported cstate change: term changed from 0 to 1, leader changed from to 5014824b94b64fd49da2c1a645c078aa (127.0.52.194). New cstate: current_term: 1 leader_uuid: "5014824b94b64fd49da2c1a645c078aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: HEALTHY } } } I0507 20:05:15.869915 607 raft_consensus.cc:1057] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Refusing update from remote peer 5014824b94b64fd49da2c1a645c078aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I0507 20:05:15.870039 363 raft_consensus.cc:1057] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Refusing update from remote peer 5014824b94b64fd49da2c1a645c078aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I0507 20:05:15.870764 675 consensus_queue.cc:963] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0507 20:05:15.871022 683 consensus_queue.cc:963] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0507 20:05:15.872639 679 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I0507 20:05:15.873425 363 raft_consensus.cc:1057] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Refusing update from remote peer 5014824b94b64fd49da2c1a645c078aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I0507 20:05:15.875241 683 consensus_queue.cc:963] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0507 20:05:15.882158 608 raft_consensus.cc:1057] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Refusing update from remote peer 5014824b94b64fd49da2c1a645c078aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I0507 20:05:15.883111 675 consensus_queue.cc:963] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0507 20:05:15.936283 675 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:15.936410 675 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.104s (starting election) I0507 20:05:15.936496 675 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.936920 675 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.937124 675 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.938025 361 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:15.938112 607 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:15.938236 361 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.117s (vote granted) I0507 20:05:15.938302 361 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 1. I0507 20:05:15.939162 607 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.125s (vote granted) I0507 20:05:15.939237 607 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 1. I0507 20:05:15.939740 420 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 pre-election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.939784 420 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. I0507 20:05:15.940019 420 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 pre-election: Vote granted by peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.940186 675 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.168s (election complete) I0507 20:05:15.940241 675 raft_consensus.cc:2511] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Leader pre-election won for term 2 I0507 20:05:15.940285 675 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0507 20:05:15.940322 675 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.112s (starting election) I0507 20:05:15.940366 675 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:15.942049 675 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.942373 675 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 election: Requesting vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.942487 675 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 election: Requesting vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.943126 361 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" I0507 20:05:15.943281 361 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:15.943367 361 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.168s (vote granted) I0507 20:05:15.943317 607 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" I0507 20:05:15.943470 607 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:15.943581 607 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.157s (vote granted) I0507 20:05:15.945482 361 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 2. I0507 20:05:15.946007 607 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5014824b94b64fd49da2c1a645c078aa in term 2. I0507 20:05:15.946588 420 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 election: Vote granted by peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:15.946655 420 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. I0507 20:05:15.946836 689 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.164s (election complete) I0507 20:05:15.946956 689 raft_consensus.cc:2511] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Leader election won for term 2 I0507 20:05:15.947064 689 raft_consensus.cc:571] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 2 LEADER]: Becoming Leader. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Running, Role: LEADER I0507 20:05:15.947324 689 consensus_queue.cc:204] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:15.948875 420 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 2 election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:15.951681 241 catalog_manager.cc:3716] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa reported cstate change: term changed from 0 to 2, leader changed from to 5014824b94b64fd49da2c1a645c078aa (127.0.52.194). New cstate: current_term: 2 leader_uuid: "5014824b94b64fd49da2c1a645c078aa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: UNKNOWN } } } I0507 20:05:15.969058 361 raft_consensus.cc:1057] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Refusing update from remote peer 5014824b94b64fd49da2c1a645c078aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch) I0507 20:05:15.969684 675 consensus_queue.cc:963] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0507 20:05:15.970052 607 raft_consensus.cc:1057] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Refusing update from remote peer 5014824b94b64fd49da2c1a645c078aa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 2 index: 1. (index mismatch) I0507 20:05:15.972375 689 consensus_queue.cc:963] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s I0507 20:05:17.028301 679 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.028435 679 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.093s (starting election) I0507 20:05:17.028560 679 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:17.029028 679 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.029201 679 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.035099 679 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.035189 679 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.079s (starting election) I0507 20:05:17.035274 679 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:17.035523 704 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.035598 704 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.111s (starting election) I0507 20:05:17.035748 679 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.035699 704 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:17.035940 679 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.036114 704 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.036262 704 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.055104 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:17.055379 608 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.103s (vote granted) I0507 20:05:17.055461 608 raft_consensus.cc:2219] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1bf0f72a88f4483a1906bc8246849dd in term 1. I0507 20:05:17.059762 607 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:17.060257 607 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.078s (vote granted) I0507 20:05:17.060969 297 leader_election.cc:362] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.061048 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. I0507 20:05:17.061296 704 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.081s (election complete) I0507 20:05:17.061372 704 raft_consensus.cc:2511] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Leader pre-election won for term 2 I0507 20:05:17.061414 704 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.061461 704 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.086s (starting election) I0507 20:05:17.061497 704 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:17.063969 704 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:17.064414 704 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: Requesting vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.064574 704 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: Requesting vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.060472 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:17.065997 705 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.066056 608 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.110s (vote granted) I0507 20:05:17.066143 608 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1bf0f72a88f4483a1906bc8246849dd in term 2. I0507 20:05:17.066097 705 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.088s (starting election) I0507 20:05:17.066790 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" I0507 20:05:17.069732 607 raft_consensus.cc:2219] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1bf0f72a88f4483a1906bc8246849dd in term 1. I0507 20:05:17.070035 706 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.070109 706 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.105s (starting election) I0507 20:05:17.070040 705 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:17.070209 706 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:17.070513 705 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:17.070643 608 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:17.070777 608 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.110s (vote granted) I0507 20:05:17.070789 706 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.071521 297 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 pre-election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.071580 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. I0507 20:05:17.071805 704 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.088s (election complete) I0507 20:05:17.071887 704 raft_consensus.cc:2511] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Leader pre-election won for term 3 I0507 20:05:17.071928 704 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Starting leader election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.071977 704 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.080s (starting election) I0507 20:05:17.072016 704 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:17.072077 363 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:17.072304 363 raft_consensus.cc:2149] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 in current term 2: Already voted for candidate d1bf0f72a88f4483a1906bc8246849dd in this term. I0507 20:05:17.073105 297 leader_election.cc:362] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.073166 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. I0507 20:05:17.073398 679 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.079s (election complete) I0507 20:05:17.073467 679 raft_consensus.cc:2511] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Leader pre-election won for term 2 I0507 20:05:17.073521 679 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Starting leader election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.073568 679 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.089s (starting election) I0507 20:05:17.073622 679 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:17.073632 542 leader_election.cc:376] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d1bf0f72a88f4483a1906bc8246849dd. Message: Invalid argument: T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 in current term 2: Already voted for candidate d1bf0f72a88f4483a1906bc8246849dd in this term. I0507 20:05:17.073757 706 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:17.073931 706 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 5014824b94b64fd49da2c1a645c078aa) I0507 20:05:17.074008 706 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.088s (starting election) I0507 20:05:17.074074 706 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:17.074112 704 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:17.074450 706 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.074467 704 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 election: Requesting vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.074561 706 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:17.074683 608 raft_consensus.cc:2219] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1bf0f72a88f4483a1906bc8246849dd in term 2. I0507 20:05:17.075047 705 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.075614 297 leader_election.cc:362] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.075667 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. I0507 20:05:17.076674 679 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:17.077589 361 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:17.077903 363 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:17.077973 361 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 2 because replica is either leader or believes a valid leader to be alive. I0507 20:05:17.078053 363 raft_consensus.cc:2149] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 in current term 3: Already voted for candidate d1bf0f72a88f4483a1906bc8246849dd in this term. I0507 20:05:17.078189 679 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: Requesting vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:17.078562 679 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: Requesting vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.078567 542 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: Vote denied by peer d1bf0f72a88f4483a1906bc8246849dd. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 2 because replica is either leader or believes a valid leader to be alive. I0507 20:05:17.078727 679 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.088s (election complete) I0507 20:05:17.078780 679 raft_consensus.cc:2511] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Leader election won for term 2 I0507 20:05:17.078860 679 raft_consensus.cc:571] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Becoming Leader. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Running, Role: LEADER I0507 20:05:17.079057 542 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Vote denied by peer d1bf0f72a88f4483a1906bc8246849dd. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 in current term 3: Already voted for candidate d1bf0f72a88f4483a1906bc8246849dd in this term. I0507 20:05:17.079197 679 consensus_queue.cc:204] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:17.080214 704 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 election: Requesting vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.081126 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" I0507 20:05:17.081275 608 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:17.081337 608 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.088s (vote granted) I0507 20:05:17.082056 242 catalog_manager.cc:3716] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd reported cstate change: term changed from 1 to 2, leader changed from 5014824b94b64fd49da2c1a645c078aa (127.0.52.194) to d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193). New cstate: current_term: 2 leader_uuid: "d1bf0f72a88f4483a1906bc8246849dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: UNKNOWN } } } I0507 20:05:17.082720 608 raft_consensus.cc:2219] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1bf0f72a88f4483a1906bc8246849dd in term 2. I0507 20:05:17.083286 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" I0507 20:05:17.083477 297 leader_election.cc:362] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.083477 608 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:17.083529 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. I0507 20:05:17.083542 608 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.083s (vote granted) I0507 20:05:17.083745 679 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.096s (election complete) I0507 20:05:17.083809 679 raft_consensus.cc:2511] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Leader election won for term 2 I0507 20:05:17.083900 679 raft_consensus.cc:571] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Becoming Leader. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Running, Role: LEADER I0507 20:05:17.084074 679 consensus_queue.cc:204] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:17.085386 608 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1bf0f72a88f4483a1906bc8246849dd in term 3. I0507 20:05:17.085973 297 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 election: Vote granted by peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:17.086056 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. I0507 20:05:17.086251 708 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.104s (election complete) I0507 20:05:17.086345 708 raft_consensus.cc:2511] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader election won for term 3 I0507 20:05:17.086716 708 raft_consensus.cc:571] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 LEADER]: Becoming Leader. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Running, Role: LEADER I0507 20:05:17.086966 708 consensus_queue.cc:204] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:17.090498 242 catalog_manager.cc:3716] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd reported cstate change: term changed from 1 to 2, leader changed from 5014824b94b64fd49da2c1a645c078aa (127.0.52.194) to d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193). New cstate: current_term: 2 leader_uuid: "d1bf0f72a88f4483a1906bc8246849dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: UNKNOWN } } } I0507 20:05:17.090936 242 catalog_manager.cc:3716] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd reported cstate change: term changed from 2 to 3, leader changed from 5014824b94b64fd49da2c1a645c078aa (127.0.52.194) to d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193). New cstate: current_term: 3 leader_uuid: "d1bf0f72a88f4483a1906bc8246849dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: UNKNOWN } } } I0507 20:05:17.105494 608 raft_consensus.cc:1057] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Refusing update from remote peer d1bf0f72a88f4483a1906bc8246849dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch) I0507 20:05:17.106297 704 consensus_queue.cc:963] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s I0507 20:05:17.108342 607 raft_consensus.cc:1057] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Refusing update from remote peer d1bf0f72a88f4483a1906bc8246849dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch) I0507 20:05:17.109252 679 consensus_queue.cc:963] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s I0507 20:05:17.109571 608 raft_consensus.cc:1057] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Refusing update from remote peer d1bf0f72a88f4483a1906bc8246849dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch) I0507 20:05:17.110205 708 consensus_queue.cc:963] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s W0507 20:05:17.115999 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.079s (SENT) W0507 20:05:17.122359 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.093s (SENT) W0507 20:05:17.148422 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.111s (SENT) W0507 20:05:17.151844 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.086s (SENT) W0507 20:05:17.156257 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.080s (SENT) W0507 20:05:17.164803 541 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.088s (SENT) I0507 20:05:17.164908 541 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. I0507 20:05:17.165161 707 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.095s (election complete) I0507 20:05:17.165248 707 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority W0507 20:05:17.165295 541 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.088s (SENT) I0507 20:05:17.165341 541 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. I0507 20:05:17.165493 707 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.110s (election complete) I0507 20:05:17.165554 707 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority W0507 20:05:17.168004 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.089s (SENT) W0507 20:05:17.181025 541 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.105s (SENT) I0507 20:05:17.181114 541 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. I0507 20:05:17.181370 707 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.096s (election complete) I0507 20:05:17.181438 707 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority W0507 20:05:19.036597 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:16.037889 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.194:43942 0507 20:05:16.038106 (+ 217us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:16.038121 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:16.038231 (+ 110us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:19.035180 (+2996949us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.194:43942 Metrics: {"client-negotiator.queue_time_us":57} W0507 20:05:19.036927 658 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.194:43942 W0507 20:05:22.079358 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:19.082071 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.194:43942 0507 20:05:19.082315 (+ 244us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:19.082330 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:19.082440 (+ 110us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:22.079122 (+2996682us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.194:43942 Metrics: {"client-negotiator.queue_time_us":54} W0507 20:05:22.079550 658 meta_cache.cc:216] Tablet d80f4fae4b9f4d238f7f626507290390: Replica 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.194:43942 I0507 20:05:23.135113 485 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.137821 486 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.138404 485 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 2 because replica is either leader or believes a valid leader to be alive. I0507 20:05:23.139636 608 raft_consensus.cc:1025] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Rejecting Update request from peer 5014824b94b64fd49da2c1a645c078aa for earlier term 2. Current term is 3. Ops: [] I0507 20:05:23.140202 608 raft_consensus.cc:1025] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Rejecting Update request from peer 5014824b94b64fd49da2c1a645c078aa for earlier term 1. Current term is 2. Ops: [] I0507 20:05:23.144950 486 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 2 because replica is either leader or believes a valid leader to be alive. I0507 20:05:23.147119 739 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.147233 739 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.094s (starting election) I0507 20:05:23.147405 739 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.147964 608 raft_consensus.cc:1025] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Rejecting Update request from peer 5014824b94b64fd49da2c1a645c078aa for earlier term 1. Current term is 2. Ops: [] I0507 20:05:23.148417 739 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.148624 739 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.149003 739 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.149104 739 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.152s (starting election) I0507 20:05:23.149204 739 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.149591 739 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.149541 486 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.149850 486 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 2 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 3 because replica is either leader or believes a valid leader to be alive. I0507 20:05:23.149909 739 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.149787 740 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.150135 740 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.076s (starting election) I0507 20:05:23.150255 740 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:23.150640 740 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.151039 740 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.168236 683 consensus_queue.cc:974] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: INVALID_TERM, Last received: 2.1, Next index: 2, Last known committed idx: 3, Time since last communication: 0.018s I0507 20:05:23.168743 683 raft_consensus.cc:2751] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 2 LEADER]: Stepping down as leader of term 2 I0507 20:05:23.169694 683 raft_consensus.cc:604] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 2 LEADER]: Becoming Follower/Learner. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Running, Role: LEADER I0507 20:05:23.170714 486 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" I0507 20:05:23.172232 486 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 2 because replica is either leader or believes a valid leader to be alive. I0507 20:05:23.173617 485 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.171456 683 consensus_queue.cc:227] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 2.1, Last appended by leader: 1, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.174275 689 consensus_queue.cc:974] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 4, Time since last communication: 0.001s I0507 20:05:23.174765 486 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.176700 486 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 2 because replica is either leader or believes a valid leader to be alive. I0507 20:05:23.176867 484 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.183840 683 consensus_peers.cc:456] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa -> Peer b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453): Closing peer: b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:23.183955 683 consensus_peers.cc:456] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Closing peer: d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.176609 483 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 3 candidate_status { last_received { term: 2 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" I0507 20:05:23.185611 486 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.186033 683 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:23.186050 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.186583 482 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is either leader or believes a valid leader to be alive. I0507 20:05:23.186192 481 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.186290 480 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 2 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" I0507 20:05:23.187474 541 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5014824b94b64fd49da2c1a645c078aa. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is either leader or believes a valid leader to be alive. I0507 20:05:23.187683 480 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 2 because replica is either leader or believes a valid leader to be alive. W0507 20:05:23.195045 743 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:23.135286 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:50986 0507 20:05:23.171135 (+ 35849us) server_negotiation.cc:176] Beginning negotiation 0507 20:05:23.171139 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:05:23.171157 (+ 18us) server_negotiation.cc:373] Connection header received 0507 20:05:23.171233 (+ 76us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:05:23.171236 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:05:23.171314 (+ 78us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:05:23.174529 (+ 3215us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50986: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":35697,"thread_start_us":35347,"threads_started":1} I0507 20:05:23.189558 482 raft_consensus.cc:2751] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Stepping down as leader of term 1 W0507 20:05:23.196074 742 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:23.137707 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:50989 0507 20:05:23.171903 (+ 34196us) server_negotiation.cc:176] Beginning negotiation 0507 20:05:23.171907 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:05:23.171920 (+ 13us) server_negotiation.cc:373] Connection header received 0507 20:05:23.171976 (+ 56us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:05:23.171980 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:05:23.172054 (+ 74us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:05:23.176083 (+ 4029us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50989: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":34079,"thread_start_us":33856,"threads_started":1} I0507 20:05:23.197276 683 consensus_queue.cc:974] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: INVALID_TERM, Last received: 1.1, Next index: 2, Last known committed idx: 2, Time since last communication: 0.000s I0507 20:05:23.198076 485 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.100s (vote granted) I0507 20:05:23.198150 485 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 3. I0507 20:05:23.198428 483 raft_consensus.cc:2194] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 3 because replica is already servicing an update from a current leader or another vote. I0507 20:05:23.198982 481 raft_consensus.cc:2194] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 4 because replica is already servicing an update from a current leader or another vote. I0507 20:05:23.200268 541 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: Vote denied by peer 5014824b94b64fd49da2c1a645c078aa. Message: Service unavailable: T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 4 because replica is already servicing an update from a current leader or another vote. I0507 20:05:23.200393 482 raft_consensus.cc:604] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Running, Role: LEADER I0507 20:05:23.200595 481 raft_consensus.cc:1057] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Refusing update from remote peer d1bf0f72a88f4483a1906bc8246849dd: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 3 index: 2. (index mismatch) I0507 20:05:23.202921 484 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 2 because replica is either leader or believes a valid leader to be alive. I0507 20:05:23.202764 482 consensus_queue.cc:227] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.203244 486 raft_consensus.cc:2194] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is already servicing an update from a current leader or another vote. I0507 20:05:23.203245 482 consensus_peers.cc:456] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa -> Peer b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453): Closing peer: b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:23.203415 675 raft_consensus.cc:2751] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Stepping down as leader of term 1 I0507 20:05:23.203986 675 raft_consensus.cc:604] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 5014824b94b64fd49da2c1a645c078aa, State: Running, Role: LEADER I0507 20:05:23.204011 541 leader_election.cc:376] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Vote denied by peer 5014824b94b64fd49da2c1a645c078aa. Message: Service unavailable: T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is already servicing an update from a current leader or another vote. I0507 20:05:23.204278 675 consensus_queue.cc:227] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 1, Committed index: 1, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:23.204495 675 consensus_peers.cc:456] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Closing peer: d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.204550 675 consensus_peers.cc:456] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa -> Peer b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453): Closing peer: b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:23.204711 675 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:23.207108 482 consensus_peers.cc:456] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Closing peer: d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.207260 482 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 1 FOLLOWER]: Advancing to term 2 I0507 20:05:23.208528 480 raft_consensus.cc:1057] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Refusing update from remote peer d1bf0f72a88f4483a1906bc8246849dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch) I0507 20:05:23.210053 482 raft_consensus.cc:1057] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Refusing update from remote peer d1bf0f72a88f4483a1906bc8246849dd: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 2 index: 2. (index mismatch) W0507 20:05:23.210166 683 raft_consensus.cc:746] Couldn't advance consensus term.: Illegal state: Can't advance term to: 2 current term: 2 is higher. I0507 20:05:23.223737 744 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Starting pre-election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.223845 744 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.163s (starting election) I0507 20:05:23.223932 744 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.224356 744 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.224548 744 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.225339 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.225518 482 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.091s (vote granted) I0507 20:05:23.225572 482 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 3. I0507 20:05:23.225992 541 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: Vote granted by peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.226066 541 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate won. I0507 20:05:23.226279 744 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.127s (election complete) I0507 20:05:23.226351 744 raft_consensus.cc:2511] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Leader pre-election won for term 4 I0507 20:05:23.226393 744 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Starting leader election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.226461 744 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.145s (starting election) I0507 20:05:23.226503 744 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Advancing to term 4 W0507 20:05:23.227143 542 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.076s (SENT) I0507 20:05:23.227207 542 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. I0507 20:05:23.228243 744 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.228555 744 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 election: Requesting vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.228696 744 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 election: Requesting vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.228854 744 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.111s (election complete) I0507 20:05:23.229013 744 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority I0507 20:05:23.229315 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" I0507 20:05:23.229454 482 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Advancing to term 4 I0507 20:05:23.229528 482 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.080s (vote granted) I0507 20:05:23.231155 482 raft_consensus.cc:2219] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 4. I0507 20:05:23.231597 541 leader_election.cc:362] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 election: Vote granted by peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.231663 541 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. I0507 20:05:23.231842 744 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.076s (election complete) I0507 20:05:23.231906 744 raft_consensus.cc:2511] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 FOLLOWER]: Leader election won for term 4 I0507 20:05:23.232004 744 raft_consensus.cc:571] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Becoming Leader. State: Replica: b5e887ddd8a14cf3b2c24a0aa14be025, State: Running, Role: LEADER I0507 20:05:23.233201 744 consensus_queue.cc:204] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.236193 241 catalog_manager.cc:3716] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 reported cstate change: term changed from 3 to 4, leader changed from d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193) to b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195). New cstate: current_term: 4 leader_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: HEALTHY } } } I0507 20:05:23.242419 744 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.242540 744 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.097s (starting election) I0507 20:05:23.242633 744 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.243106 744 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.243265 744 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd W0507 20:05:23.243535 542 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.094s (SENT) I0507 20:05:23.243613 542 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. I0507 20:05:23.243795 744 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.164s (election complete) I0507 20:05:23.243851 744 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority I0507 20:05:23.244108 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.244277 482 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.098s (vote granted) I0507 20:05:23.244343 482 raft_consensus.cc:2219] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 2. I0507 20:05:23.244850 541 leader_election.cc:362] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Vote granted by peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.244915 541 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. I0507 20:05:23.245162 744 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.110s (election complete) I0507 20:05:23.245234 744 raft_consensus.cc:2511] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election won for term 3 I0507 20:05:23.245293 744 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.245342 744 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.116s (starting election) I0507 20:05:23.245401 744 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:23.247334 744 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.247685 744 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: Requesting vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.247833 744 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: Requesting vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.248584 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" I0507 20:05:23.248750 482 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:23.248831 482 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.096s (vote granted) I0507 20:05:23.250439 482 raft_consensus.cc:2219] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 3. I0507 20:05:23.250985 541 leader_election.cc:362] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: Vote granted by peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.251054 541 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. I0507 20:05:23.251281 744 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.108s (election complete) I0507 20:05:23.251348 744 raft_consensus.cc:2511] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Leader election won for term 3 I0507 20:05:23.251497 744 raft_consensus.cc:571] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Becoming Leader. State: Replica: b5e887ddd8a14cf3b2c24a0aa14be025, State: Running, Role: LEADER I0507 20:05:23.251703 744 consensus_queue.cc:204] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:23.254166 242 catalog_manager.cc:3716] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 reported cstate change: term changed from 2 to 3, leader changed from d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193) to b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195). New cstate: current_term: 3 leader_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: HEALTHY } } } I0507 20:05:23.257102 482 raft_consensus.cc:1057] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Refusing update from remote peer b5e887ddd8a14cf3b2c24a0aa14be025: Log matching property violated. Preceding OpId in replica: term: 2 index: 1. Preceding OpId from leader: term: 4 index: 4. (index mismatch) I0507 20:05:23.257773 744 consensus_queue.cc:963] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 1, Time since last communication: 0.000s I0507 20:05:23.268942 683 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0507 20:05:23.269038 683 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.111s (starting election) I0507 20:05:23.269119 683 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:23.269549 683 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:23.269693 683 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.270491 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:23.270743 608 raft_consensus.cc:2166] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1. I0507 20:05:23.271287 420 leader_election.cc:376] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 3 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 3 because replica has last-logged OpId of term: 2 index: 4, which is greater than that of the candidate, which has last-logged OpId of term: 1 index: 1. I0507 20:05:23.281529 482 raft_consensus.cc:1057] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Refusing update from remote peer b5e887ddd8a14cf3b2c24a0aa14be025: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 3. (index mismatch) I0507 20:05:23.282232 744 consensus_queue.cc:963] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 1, Time since last communication: 0.000s W0507 20:05:23.302737 542 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.152s (SENT) I0507 20:05:23.302829 542 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:23.303081 740 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.099s (election complete) I0507 20:05:23.303174 740 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:23.340993 740 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.341121 740 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.248s (starting election) W0507 20:05:23.341308 542 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.097s (SENT) I0507 20:05:23.341210 740 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:23.341686 740 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.341837 740 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.342592 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:23.342759 482 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.090s (vote granted) I0507 20:05:23.342813 482 raft_consensus.cc:2219] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 2. I0507 20:05:23.343256 541 leader_election.cc:362] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Vote granted by peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.343324 541 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. I0507 20:05:23.343535 740 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.108s (election complete) I0507 20:05:23.343595 740 raft_consensus.cc:2511] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Leader pre-election won for term 3 I0507 20:05:23.343636 740 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Starting leader election (detected failure of leader d1bf0f72a88f4483a1906bc8246849dd) I0507 20:05:23.343667 740 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.126s (starting election) I0507 20:05:23.343708 740 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:23.345487 740 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:23.345829 740 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: Requesting vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:23.345943 740 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: Requesting vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.346683 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" I0507 20:05:23.346853 482 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:23.346958 482 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.102s (vote granted) I0507 20:05:23.348662 482 raft_consensus.cc:2219] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 3. I0507 20:05:23.349109 541 leader_election.cc:362] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: Vote granted by peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:23.349177 541 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. I0507 20:05:23.349393 740 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Snoozing failure detection for 0.086s (election complete) I0507 20:05:23.349474 740 raft_consensus.cc:2511] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 FOLLOWER]: Leader election won for term 3 I0507 20:05:23.349586 740 raft_consensus.cc:571] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Becoming Leader. State: Replica: b5e887ddd8a14cf3b2c24a0aa14be025, State: Running, Role: LEADER I0507 20:05:23.349793 740 consensus_queue.cc:204] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:23.352468 242 catalog_manager.cc:3716] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 reported cstate change: term changed from 2 to 3, leader changed from d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193) to b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195). New cstate: current_term: 3 leader_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } health_report { overall_health: UNKNOWN } } } W0507 20:05:23.364323 542 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.116s (SENT) W0507 20:05:23.375610 542 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.145s (SENT) I0507 20:05:23.377871 482 raft_consensus.cc:1057] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Refusing update from remote peer b5e887ddd8a14cf3b2c24a0aa14be025: Log matching property violated. Preceding OpId in replica: term: 1 index: 1. Preceding OpId from leader: term: 3 index: 5. (index mismatch) I0507 20:05:23.378530 740 consensus_queue.cc:963] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 1, Time since last communication: 0.000s W0507 20:05:23.381114 420 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.111s (SENT) I0507 20:05:23.381206 420 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate lost. I0507 20:05:23.383507 683 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.109s (election complete) I0507 20:05:23.383589 683 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election lost for term 3. Reason: could not achieve majority W0507 20:05:23.388511 542 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.163s (SENT) W0507 20:05:23.472944 542 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.126s (SENT) W0507 20:05:23.590140 542 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.248s (SENT) W0507 20:05:26.219467 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:23.219696 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.193:46057 0507 20:05:23.219973 (+ 277us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:23.219987 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:23.220097 (+ 110us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:26.219176 (+2999079us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.193:46057 Metrics: {"client-negotiator.queue_time_us":74} W0507 20:05:26.219700 659 meta_cache.cc:216] Tablet 7161aaf1a71d455883ef478337dfaf19: Replica d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.193:46057 W0507 20:05:29.239563 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:26.241294 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.193:46057 0507 20:05:26.241523 (+ 229us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:26.241538 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:26.241644 (+ 106us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:29.239179 (+2997535us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.193:46057 Metrics: {"client-negotiator.queue_time_us":64} W0507 20:05:29.239842 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.193:46057 W0507 20:05:32.263381 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:29.264048 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.193:46057 0507 20:05:29.264349 (+ 301us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:29.264362 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:29.264480 (+ 118us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:32.263118 (+2998638us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.193:46057 Metrics: {"client-negotiator.queue_time_us":130} W0507 20:05:32.263617 659 meta_cache.cc:216] Tablet d80f4fae4b9f4d238f7f626507290390: Replica d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.193:46057 I0507 20:05:33.317754 360 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 3 candidate_status { last_received { term: 1 index: 1 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:33.318048 360 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 3 because replica is either leader or believes a valid leader to be alive. I0507 20:05:33.318599 361 raft_consensus.cc:1025] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Rejecting Update request from peer 5014824b94b64fd49da2c1a645c078aa for earlier term 1. Current term is 2. Ops: [] I0507 20:05:33.319263 362 raft_consensus.cc:1025] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Rejecting Update request from peer 5014824b94b64fd49da2c1a645c078aa for earlier term 1. Current term is 2. Ops: [] I0507 20:05:33.319880 733 consensus_queue.cc:963] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s I0507 20:05:33.321444 733 consensus_queue.cc:963] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.001s I0507 20:05:33.338580 363 raft_consensus.cc:1025] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 LEADER]: Rejecting Update request from peer 5014824b94b64fd49da2c1a645c078aa for earlier term 2. Current term is 3. Ops: [] I0507 20:05:33.348132 733 consensus_queue.cc:963] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 2, Last known committed idx: 1, Time since last communication: 0.000s W0507 20:05:33.349349 791 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:33.319584 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:59090 0507 20:05:33.347901 (+ 28317us) server_negotiation.cc:176] Beginning negotiation 0507 20:05:33.347906 (+ 5us) server_negotiation.cc:365] Waiting for connection header 0507 20:05:33.347925 (+ 19us) server_negotiation.cc:373] Connection header received 0507 20:05:33.347984 (+ 59us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:05:33.347988 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:05:33.348076 (+ 88us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:05:33.348296 (+ 220us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59090: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":26784,"threads_started":1} I0507 20:05:33.350270 363 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:33.350515 363 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is either leader or believes a valid leader to be alive. I0507 20:05:33.350565 608 raft_consensus.cc:1025] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Rejecting Update request from peer d1bf0f72a88f4483a1906bc8246849dd for earlier term 3. Current term is 4. Ops: [] I0507 20:05:33.351606 606 raft_consensus.cc:1025] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Rejecting Update request from peer d1bf0f72a88f4483a1906bc8246849dd for earlier term 2. Current term is 3. Ops: [] I0507 20:05:33.352030 608 raft_consensus.cc:1025] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Rejecting Update request from peer d1bf0f72a88f4483a1906bc8246849dd for earlier term 2. Current term is 3. Ops: [] W0507 20:05:33.352203 791 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:33.349107 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:59086 0507 20:05:33.349636 (+ 529us) server_negotiation.cc:176] Beginning negotiation 0507 20:05:33.349639 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:05:33.349651 (+ 12us) server_negotiation.cc:373] Connection header received 0507 20:05:33.349690 (+ 39us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:05:33.349693 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:05:33.349777 (+ 84us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:05:33.349886 (+ 109us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59086: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":241,"threads_started":1} W0507 20:05:33.352288 788 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:33.347433 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:59088 0507 20:05:33.348409 (+ 976us) server_negotiation.cc:176] Beginning negotiation 0507 20:05:33.348412 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:05:33.348428 (+ 16us) server_negotiation.cc:373] Connection header received 0507 20:05:33.348463 (+ 35us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:05:33.348467 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:05:33.348528 (+ 61us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:05:33.348680 (+ 152us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59088: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":861,"thread_start_us":162,"threads_started":1} I0507 20:05:33.352596 733 consensus_queue.cc:974] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: INVALID_TERM, Last received: 3.3, Next index: 4, Last known committed idx: 5, Time since last communication: 0.000s I0507 20:05:33.352769 786 consensus_queue.cc:974] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Peer responded invalid term: Peer: permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 }, Status: INVALID_TERM, Last received: 2.4, Next index: 5, Last known committed idx: 6, Time since last communication: 0.000s I0507 20:05:33.353931 786 consensus_queue.cc:1102] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Peer b5e887ddd8a14cf3b2c24a0aa14be025 log is divergent from this leader: its last log entry 3.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 6 I0507 20:05:33.352773 363 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:33.354075 785 raft_consensus.cc:2751] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Stepping down as leader of term 2 I0507 20:05:33.354187 785 raft_consensus.cc:604] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Running, Role: LEADER I0507 20:05:33.355439 362 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:33.354476 785 consensus_queue.cc:227] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 2.4, Last appended by leader: 4, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:33.355868 785 consensus_peers.cc:456] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Closing peer: 5014824b94b64fd49da2c1a645c078aa I0507 20:05:33.356286 785 consensus_peers.cc:456] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd -> Peer b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453): Closing peer: b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:33.356628 785 raft_consensus.cc:2756] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:33.357547 360 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" I0507 20:05:33.357851 361 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 4 candidate_status { last_received { term: 3 index: 3 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:33.359174 358 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" I0507 20:05:33.356323 359 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 2 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:33.359448 355 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:33.359858 356 raft_consensus.cc:2751] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Stepping down as leader of term 2 I0507 20:05:33.359925 356 raft_consensus.cc:604] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 LEADER]: Becoming Follower/Learner. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Running, Role: LEADER I0507 20:05:33.360329 356 consensus_queue.cc:227] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 2.2, Last appended by leader: 2, Current term: 2, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:33.361647 356 consensus_peers.cc:456] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd -> Peer b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453): Closing peer: b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:33.361284 354 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" candidate_term: 3 candidate_status { last_received { term: 2 index: 4 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" I0507 20:05:33.361888 356 consensus_peers.cc:456] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Closing peer: 5014824b94b64fd49da2c1a645c078aa I0507 20:05:33.362156 356 raft_consensus.cc:2756] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 2 FOLLOWER]: Advancing to term 3 I0507 20:05:33.362352 733 consensus_queue.cc:1102] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [LEADER]: Peer b5e887ddd8a14cf3b2c24a0aa14be025 log is divergent from this leader: its last log entry 4.6 is not in this leader's log and it has not received anything from this leader yet. Falling back to committed index 5 I0507 20:05:33.362948 363 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:33.363258 360 raft_consensus.cc:2194] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 LEADER]: Leader election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 4 because replica is already servicing an update from a current leader or another vote. I0507 20:05:33.363713 357 raft_consensus.cc:2751] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 LEADER]: Stepping down as leader of term 3 I0507 20:05:33.363787 357 raft_consensus.cc:604] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 LEADER]: Becoming Follower/Learner. State: Replica: d1bf0f72a88f4483a1906bc8246849dd, State: Running, Role: LEADER I0507 20:05:33.364279 357 consensus_queue.cc:227] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 3, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 3, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:33.364724 357 consensus_peers.cc:456] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd -> Peer b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453): Closing peer: b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:33.364785 357 consensus_peers.cc:456] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Closing peer: 5014824b94b64fd49da2c1a645c078aa I0507 20:05:33.364852 362 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.101s (vote granted) I0507 20:05:33.364887 357 raft_consensus.cc:2756] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Advancing to term 4 I0507 20:05:33.364917 362 raft_consensus.cc:2219] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5e887ddd8a14cf3b2c24a0aa14be025 in term 3. I0507 20:05:33.365119 355 raft_consensus.cc:2194] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is already servicing an update from a current leader or another vote. I0507 20:05:33.365558 354 raft_consensus.cc:2194] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is already servicing an update from a current leader or another vote. I0507 20:05:33.365931 353 raft_consensus.cc:1057] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Refusing update from remote peer b5e887ddd8a14cf3b2c24a0aa14be025: Log matching property violated. Preceding OpId in replica: term: 2 index: 4. Preceding OpId from leader: term: 3 index: 5. (index mismatch) I0507 20:05:33.366497 356 raft_consensus.cc:1057] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Refusing update from remote peer b5e887ddd8a14cf3b2c24a0aa14be025: Log matching property violated. Preceding OpId in replica: term: 2 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch) I0507 20:05:33.367693 357 raft_consensus.cc:1057] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Refusing update from remote peer b5e887ddd8a14cf3b2c24a0aa14be025: Log matching property violated. Preceding OpId in replica: term: 3 index: 3. Preceding OpId from leader: term: 4 index: 4. (index mismatch) W0507 20:05:33.367892 786 raft_consensus.cc:746] Couldn't advance consensus term.: Illegal state: Can't advance term to: 4 current term: 4 is higher. I0507 20:05:33.368818 358 raft_consensus.cc:2194] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is already servicing an update from a current leader or another vote. I0507 20:05:33.368831 361 raft_consensus.cc:2194] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 4 because replica is already servicing an update from a current leader or another vote. I0507 20:05:33.369074 359 raft_consensus.cc:2194] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b5e887ddd8a14cf3b2c24a0aa14be025 for term 3 because replica is already servicing an update from a current leader or another vote. I0507 20:05:33.369889 740 consensus_queue.cc:963] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 4, Time since last communication: 0.000s I0507 20:05:33.370363 740 consensus_queue.cc:963] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s I0507 20:05:33.370672 740 consensus_queue.cc:963] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 4, Last known committed idx: 3, Time since last communication: 0.000s I0507 20:05:34.452628 806 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:34.453202 806 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.094s (starting election) I0507 20:05:34.453341 806 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:34.453927 806 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:34.454141 806 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:34.454300 806 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:34.454380 806 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.152s (starting election) I0507 20:05:34.454460 806 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:34.454778 806 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:34.454915 806 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:34.455044 806 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:34.455101 806 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.089s (starting election) I0507 20:05:34.455193 806 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:34.455458 806 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:34.455598 806 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:34.456295 417 thread.cc:554] raft [worker] (thread pool) Time spent creating pthread: real 1.040s user 0.004s sys 0.012s W0507 20:05:34.456351 417 thread.cc:521] raft [worker] (thread pool) Time spent starting thread: real 1.040s user 0.004s sys 0.012s W0507 20:05:34.462823 420 thread.cc:554] raft [worker] (thread pool) Time spent creating pthread: real 1.046s user 0.004s sys 0.004s W0507 20:05:34.462929 420 thread.cc:521] raft [worker] (thread pool) Time spent starting thread: real 1.046s user 0.004s sys 0.004s I0507 20:05:34.464130 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:34.464365 608 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:34.465667 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:34.465837 608 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:34.465997 606 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:34.466286 606 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:34.470232 482 raft_consensus.cc:1002] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Deduplicated request from leader. Original: 2.1->[3.2-3.3] Dedup: 3.3->[] I0507 20:05:34.470823 482 raft_consensus.cc:1025] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Rejecting Update request from peer d1bf0f72a88f4483a1906bc8246849dd for earlier term 3. Current term is 4. Ops: [3.2-3.3] I0507 20:05:34.478137 420 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:34.479208 482 raft_consensus.cc:1002] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-2.4] Dedup: 2.4->[] I0507 20:05:34.479905 482 raft_consensus.cc:1025] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Rejecting Update request from peer d1bf0f72a88f4483a1906bc8246849dd for earlier term 2. Current term is 3. Ops: [2.2-2.4] W0507 20:05:34.480247 418 thread.cc:554] raft [worker] (thread pool) Time spent creating pthread: real 1.055s user 0.004s sys 0.008s W0507 20:05:34.480334 418 thread.cc:521] raft [worker] (thread pool) Time spent starting thread: real 1.064s user 0.004s sys 0.008s I0507 20:05:34.480393 420 leader_election.cc:376] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:34.481042 482 raft_consensus.cc:1002] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.1->[2.2-2.2] Dedup: 2.2->[] I0507 20:05:34.481128 482 raft_consensus.cc:1025] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Rejecting Update request from peer d1bf0f72a88f4483a1906bc8246849dd for earlier term 2. Current term is 3. Ops: [2.2-2.2] I0507 20:05:34.482152 420 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:35.568154 786 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:35.568300 786 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.109s (starting election) I0507 20:05:35.568395 786 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:35.570822 733 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:35.572201 785 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:35.572576 733 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.105s (starting election) I0507 20:05:35.572813 359 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 4 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:35.573438 785 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.085s (starting election) I0507 20:05:35.573850 733 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:35.574213 786 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:35.574786 358 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:35.575366 359 raft_consensus.cc:2166] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica has last-logged OpId of term: 3 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 3 index: 4. I0507 20:05:35.575731 733 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:35.575835 733 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:35.575915 356 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 5 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:35.576380 786 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:35.576711 358 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:35.576879 785 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:35.577213 785 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:35.578256 606 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:35.578462 606 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:35.578794 785 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:35.579272 356 raft_consensus.cc:2194] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 5 because replica is already servicing an update from a current leader or another vote. I0507 20:05:35.581181 606 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:35.581378 606 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:35.584628 297 leader_election.cc:376] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:35.591488 297 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:35.595032 606 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:35.596204 606 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:35.597538 297 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 5 because replica is either leader or believes a valid leader to be alive. W0507 20:05:35.639009 420 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.089s (SENT) I0507 20:05:35.639442 420 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:35.638790 808 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:35.639330 807 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:35.640544 808 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.087s (starting election) I0507 20:05:35.640920 807 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.131s (starting election) I0507 20:05:35.641901 808 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:35.643298 808 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:35.642153 807 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:35.644771 807 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:35.645370 808 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:35.645665 807 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd W0507 20:05:36.671355 486 thread.cc:554] prepare [worker] (thread pool) Time spent creating pthread: real 1.026s user 0.000s sys 0.000s W0507 20:05:36.671449 486 thread.cc:521] prepare [worker] (thread pool) Time spent starting thread: real 1.026s user 0.000s sys 0.000s W0507 20:05:36.702807 420 thread.cc:554] raft [worker] (thread pool) Time spent creating pthread: real 1.051s user 0.004s sys 0.008s W0507 20:05:36.707060 420 thread.cc:521] raft [worker] (thread pool) Time spent starting thread: real 1.063s user 0.004s sys 0.008s W0507 20:05:36.707970 420 outbound_call.cc:287] RPC callback for RPC call kudu.consensus.ConsensusService.RequestConsensusVote -> {remote=127.0.52.193:46057, user_credentials={real_user=slave}} blocked reactor thread for 1.07115e+06us W0507 20:05:36.712491 420 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.094s (SENT) I0507 20:05:36.712954 420 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:36.713380 808 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.086s (election complete) I0507 20:05:36.713625 486 rpcz_store.cc:265] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.52.195:50287 (request call id 1155) took 1069ms. Request Metrics: {"thread_start_us":19008,"threads_started":1,"wal-append.queue_time_us":22300,"child_traces":[["txn",{"apply.queue_time_us":21450,"apply.run_cpu_time_us":765,"apply.run_wall_time_us":1252,"num_ops":1,"prepare.queue_time_us":1177,"prepare.run_cpu_time_us":529,"prepare.run_wall_time_us":1025605,"replication_time_us":1045927,"thread_start_us":1048139,"threads_started":2}]]} I0507 20:05:36.716373 808 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:36.718973 420 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.152s (SENT) I0507 20:05:36.719306 420 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:36.721719 839 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.092s (election complete) I0507 20:05:36.721812 839 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:36.722112 807 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:36.722398 807 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.105s (starting election) I0507 20:05:36.722491 807 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:36.722976 808 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.125s (election complete) I0507 20:05:36.723067 808 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:36.723155 807 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:36.723314 807 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:36.723224 482 rpcz_store.cc:265] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.52.195:50287 (request call id 1156) took 1077ms. Request Metrics: {"thread_start_us":13292,"threads_started":1,"wal-append.queue_time_us":15085,"child_traces":[["txn",{"apply.queue_time_us":38,"apply.run_cpu_time_us":530,"apply.run_wall_time_us":530,"num_ops":1,"prepare.queue_time_us":742,"prepare.run_cpu_time_us":345,"prepare.run_wall_time_us":344,"replication_time_us":49668,"thread_start_us":35613,"threads_started":1}]]} I0507 20:05:36.724472 606 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:36.724679 606 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:36.724731 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:36.724874 608 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:36.732409 486 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 5 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:36.732641 486 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:36.733644 420 leader_election.cc:376] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:36.734380 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:36.734570 480 rpcz_store.cc:265] Call kudu.consensus.ConsensusService.UpdateConsensus from 127.0.52.195:50287 (request call id 1157) took 1088ms. Request Metrics: {"thread_start_us":19614,"threads_started":1,"wal-append.queue_time_us":31314,"child_traces":[["txn",{"apply.queue_time_us":24,"apply.run_cpu_time_us":365,"apply.run_wall_time_us":364,"num_ops":1,"prepare.queue_time_us":703,"prepare.run_cpu_time_us":132,"prepare.run_wall_time_us":132,"replication_time_us":50697,"thread_start_us":30389,"threads_started":1}]]} I0507 20:05:36.735255 608 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:36.735822 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 6 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:36.736038 482 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:36.736584 420 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:36.737149 420 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:36.737335 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 5 candidate_status { last_received { term: 4 index: 7 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:36.737557 482 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:37.800364 808 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:37.800495 808 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.105s (starting election) I0507 20:05:37.800761 808 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:37.801375 808 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:37.801563 808 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:37.802175 808 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:37.802273 808 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.076s (starting election) I0507 20:05:37.802364 808 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:37.802805 808 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:37.802986 808 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:37.803521 420 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.087s (SENT) I0507 20:05:37.803653 420 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:37.803844 808 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.134s (election complete) I0507 20:05:37.803903 808 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:37.804206 420 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.105s (SENT) I0507 20:05:37.804716 420 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:37.804962 808 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.182s (election complete) I0507 20:05:37.805074 808 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:37.805152 420 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.131s (SENT) I0507 20:05:37.805236 420 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:37.805446 808 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.189s (election complete) I0507 20:05:37.805524 808 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:37.813134 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:37.813385 608 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:37.815587 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:37.815826 608 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:37.816004 420 leader_election.cc:376] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:37.816493 420 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.897075 785 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:05:38.898300 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.085s (SENT) I0507 20:05:38.898787 733 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:38.899520 786 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:38.899575 786 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (starting election) I0507 20:05:38.899643 358 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:38.899866 361 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 7 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:38.900074 733 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.080s (starting election) I0507 20:05:38.900149 296 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:38.900635 733 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:38.900974 733 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:38.901690 733 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:38.902060 785 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.111s (starting election) I0507 20:05:38.902128 785 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:38.902962 785 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:38.903071 785 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:38.903625 793 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.077s (election complete) I0507 20:05:38.904503 793 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:38.905266 786 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:38.905598 786 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:38.906623 786 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:38.906783 361 raft_consensus.cc:2194] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is already servicing an update from a current leader or another vote. I0507 20:05:38.908241 356 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:38.909418 354 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:38.909692 354 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.909924 357 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true W0507 20:05:38.910225 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.105s (SENT) I0507 20:05:38.910305 296 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:38.910367 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:38.910650 608 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.910714 358 raft_consensus.cc:2194] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is already servicing an update from a current leader or another vote. I0507 20:05:38.911298 357 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 5 because replica is either leader or believes a valid leader to be alive. W0507 20:05:38.911629 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.109s (SENT) I0507 20:05:38.911747 296 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:38.913527 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:38.913765 608 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.914566 356 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.915184 297 leader_election.cc:376] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 4 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.915910 786 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (election complete) I0507 20:05:38.916394 786 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:38.917256 793 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.103s (election complete) I0507 20:05:38.917387 793 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:38.922797 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "a0376a5e3b9f495aa00d366b35594a27" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:38.923123 608 raft_consensus.cc:2180] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.924757 297 leader_election.cc:376] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.930755 297 leader_election.cc:376] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Vote denied by peer b5e887ddd8a14cf3b2c24a0aa14be025. Message: Invalid argument: T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. W0507 20:05:38.988833 420 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.076s (SENT) I0507 20:05:38.988929 420 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:38.991418 884 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:05:38.991497 420 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer d1bf0f72a88f4483a1906bc8246849dd: Timed out: RequestConsensusVote RPC to 127.0.52.193:46057 timed out after 0.105s (SENT) I0507 20:05:38.991576 886 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.139s (election complete) I0507 20:05:38.991619 420 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:38.991644 886 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:38.991734 885 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:38.991791 885 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.378s (starting election) I0507 20:05:38.991883 885 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:38.994133 885 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer d1bf0f72a88f4483a1906bc8246849dd I0507 20:05:38.994333 885 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:38.995613 884 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa: Snoozing failure detection for 0.100s (starting election) I0507 20:05:38.995613 480 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "d80f4fae4b9f4d238f7f626507290390" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 5 candidate_status { last_received { term: 4 index: 8 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:38.995828 480 raft_consensus.cc:2180] T d80f4fae4b9f4d238f7f626507290390 P 5014824b94b64fd49da2c1a645c078aa [term 4 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 5 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.995762 884 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:38.996187 884 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P 5014824b94b64fd49da2c1a645c078aa [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:38.996781 608 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" is_pre_election: true I0507 20:05:38.997072 608 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [term 3 LEADER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:38.997400 482 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "d1bf0f72a88f4483a1906bc8246849dd" candidate_term: 4 candidate_status { last_received { term: 3 index: 8 } } ignore_live_leader: false dest_uuid: "5014824b94b64fd49da2c1a645c078aa" is_pre_election: true I0507 20:05:38.997568 482 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P 5014824b94b64fd49da2c1a645c078aa [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate d1bf0f72a88f4483a1906bc8246849dd for term 4 because replica is either leader or believes a valid leader to be alive. I0507 20:05:40.028996 785 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:40.029045 793 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/debug-util.cc:700:42: runtime error: pointer index expression with base 0x000000000000 overflowed to 0xffffffffffffffff I0507 20:05:40.030405 785 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.153s (starting election) I0507 20:05:40.031149 786 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) SUMMARY: UndefinedBehaviorSanitizer: undefined-behavior /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/util/debug-util.cc:700:42 in I0507 20:05:40.033407 793 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.157s (starting election) W0507 20:05:40.034566 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.080s (SENT) I0507 20:05:40.035722 296 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.035946 786 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.086s (starting election) I0507 20:05:40.035495 785 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:40.036347 786 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:40.036176 793 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:40.037746 785 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:40.037884 793 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.038275 793 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:40.038784 786 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.038776 793 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.081s (election complete) I0507 20:05:40.038949 786 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:40.039649 785 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.038347 356 tablet_service.cc:998] Received RequestConsensusVote() RPC: tablet_id: "7161aaf1a71d455883ef478337dfaf19" candidate_uuid: "5014824b94b64fd49da2c1a645c078aa" candidate_term: 4 candidate_status { last_received { term: 3 index: 12 } } ignore_live_leader: false dest_uuid: "d1bf0f72a88f4483a1906bc8246849dd" is_pre_election: true I0507 20:05:40.039999 793 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:40.040509 356 raft_consensus.cc:2180] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 5014824b94b64fd49da2c1a645c078aa for term 4 because replica is either leader or believes a valid leader to be alive. W0507 20:05:40.040565 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.099s (SENT) I0507 20:05:40.040760 296 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.041072 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.111s (election complete) I0507 20:05:40.041206 894 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:40.041216 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Timed out: RequestConsensusVote RPC to 127.0.52.194:43942 timed out after 0.111s (SENT) I0507 20:05:40.041445 296 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.041657 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.081s (election complete) I0507 20:05:40.041714 894 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:40.052222 295 connection.cc:512] server connection from 127.0.52.194:43585 recv error: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Connection reset by peer (error 104) W0507 20:05:40.052299 227 connection.cc:512] server connection from 127.0.52.194:58584 recv error: Network error: failed to read from TLS socket (remote: 127.0.52.194:58584): Connection reset by peer (error 104) W0507 20:05:40.052474 658 connection.cc:512] client connection to 127.0.52.194:43942 recv error: Network error: failed to read from TLS socket (remote: 127.0.52.194:43942): Connection reset by peer (error 104) W0507 20:05:40.052652 295 connection.cc:185] Error closing socket: Network error: TlsSocket::Close: Broken pipe W0507 20:05:40.052883 226 connection.cc:512] client connection to 127.0.52.194:43942 recv error: Network error: failed to read from TLS socket (remote: 127.0.52.194:43942): Connection reset by peer (error 104) W0507 20:05:40.052934 296 connection.cc:512] client connection to 127.0.52.194:43942 recv error: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Connection reset by peer (error 104) W0507 20:05:40.053089 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Connection reset by peer (error 104) W0507 20:05:40.053205 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Connection reset by peer (error 104) W0507 20:05:40.053256 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Connection reset by peer (error 104) W0507 20:05:40.053455 296 connection.cc:185] Error closing socket: Network error: TlsSocket::Close: Broken pipe I0507 20:05:40.120949 894 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:40.121057 894 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.176s (starting election) I0507 20:05:40.121177 894 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:40.121779 894 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.121982 894 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:40.123364 894 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:05:40.123363 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:40.123453 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.230s (starting election) I0507 20:05:40.123529 894 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:40.123862 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.123984 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:40.124545 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:40.125643 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.086s (SENT) I0507 20:05:40.125723 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.125986 894 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.081s (election complete) I0507 20:05:40.126082 894 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:40.153045 894 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:40.153165 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.131s (starting election) I0507 20:05:40.153275 894 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:40.153710 894 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:40.153906 894 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:40.154996 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:40.193265 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.153s (SENT) I0507 20:05:40.193368 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.193774 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.142s (election complete) I0507 20:05:40.193876 894 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:40.196032 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.157s (SENT) I0507 20:05:40.196110 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.196456 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.243s (election complete) I0507 20:05:40.196591 894 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:40.207883 894 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:40.208052 894 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.102s (starting election) I0507 20:05:40.208168 894 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:40.208695 894 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.208899 894 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:40.210002 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:40.285490 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.131s (SENT) I0507 20:05:40.285614 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.286083 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.147s (election complete) I0507 20:05:40.286223 894 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:40.299134 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.176s (SENT) I0507 20:05:40.299257 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.299695 894 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.107s (election complete) I0507 20:05:40.299854 894 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:40.312201 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.102s (SENT) I0507 20:05:40.312355 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.312844 894 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.558s (election complete) I0507 20:05:40.312980 894 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:40.355401 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.230s (SENT) I0507 20:05:40.355588 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.356050 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.308s (election complete) I0507 20:05:40.356186 894 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:40.434051 894 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:40.434166 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.398s (starting election) I0507 20:05:40.434283 894 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:40.434756 894 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:40.435030 894 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:40.435997 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:40.664913 894 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:40.665050 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.457s (starting election) I0507 20:05:40.665148 894 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:40.665628 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.665810 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:40.666743 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:40.832691 894 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:40.832830 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.459s (starting election) I0507 20:05:40.832960 894 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:40.833650 894 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:40.833791 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.398s (SENT) I0507 20:05:40.833890 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:40.833892 894 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.834085 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.547s (election complete) I0507 20:05:40.834220 894 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:40.834851 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:40.871779 896 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:40.871908 896 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.154s (starting election) I0507 20:05:40.872068 896 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:40.872545 896 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:40.872807 896 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:40.873972 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:41.027122 896 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:41.027290 896 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.221s (starting election) I0507 20:05:41.027457 896 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:41.027741 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.154s (SENT) I0507 20:05:41.027856 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:41.027990 896 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:41.028383 896 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:41.028395 894 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.488s (election complete) I0507 20:05:41.028566 894 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:41.029291 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:41.123236 894 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:41.123385 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.152s (starting election) I0507 20:05:41.123489 894 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:41.123838 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.457s (SENT) I0507 20:05:41.123916 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:41.124133 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:41.124178 896 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.366s (election complete) I0507 20:05:41.124305 896 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:41.124347 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:41.125208 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:41.250449 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.221s (SENT) I0507 20:05:41.250564 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:41.250988 894 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.339s (election complete) I0507 20:05:41.251073 894 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:41.277520 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.152s (SENT) I0507 20:05:41.277654 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:41.278070 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.626s (election complete) I0507 20:05:41.278179 894 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:41.294970 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.459s (SENT) I0507 20:05:41.295069 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:41.295410 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.436s (election complete) I0507 20:05:41.295500 894 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:41.591676 894 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:41.591850 894 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.943s (starting election) I0507 20:05:41.591982 894 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:41.592607 894 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:41.592875 894 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:41.593636 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:41.732866 894 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:41.733012 894 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.792s (starting election) I0507 20:05:41.733125 894 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:41.733672 894 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:41.733850 894 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:41.734911 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:41.904878 894 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:41.905045 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.101s (starting election) I0507 20:05:41.905179 894 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:41.905695 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:41.905925 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:41.906960 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:42.006933 894 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:42.007129 894 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.207s (starting election) I0507 20:05:42.007272 894 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:42.007710 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.101s (SENT) I0507 20:05:42.007843 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:42.008033 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:42.008388 894 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:42.008570 897 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.379s (election complete) I0507 20:05:42.008651 897 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:42.009268 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:42.019687 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:39.021506 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:05:39.021781 (+ 275us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:39.021797 (+ 16us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:39.021942 (+ 145us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:42.019330 (+2997388us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":78} W0507 20:05:42.020074 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:05:42.216768 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.207s (SENT) I0507 20:05:42.216881 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:42.217329 897 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.222s (election complete) I0507 20:05:42.217432 897 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:42.440784 897 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:42.440904 897 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.829s (starting election) I0507 20:05:42.441012 897 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:42.441503 897 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:42.441746 897 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:42.442492 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:42.525792 897 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:42.525974 897 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.411s (starting election) I0507 20:05:42.526152 897 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:42.526638 897 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:42.526816 897 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:42.526942 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.792s (SENT) I0507 20:05:42.527034 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:42.527323 897 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.358s (election complete) I0507 20:05:42.527417 897 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:42.527990 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:42.536368 897 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:42.536473 897 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.451s (starting election) I0507 20:05:42.536571 897 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:42.536988 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.943s (SENT) I0507 20:05:42.537030 897 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:42.537101 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:42.537255 897 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:42.537384 897 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.576s (election complete) I0507 20:05:42.537451 897 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:42.538096 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:42.886960 899 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:42.887095 899 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.817s (starting election) I0507 20:05:42.887229 899 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:42.887708 899 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:42.887914 899 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:42.888975 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:42.939615 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.411s (SENT) I0507 20:05:42.939743 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:42.940145 899 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.870s (election complete) I0507 20:05:42.940270 899 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:42.989506 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.451s (SENT) I0507 20:05:42.989624 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:42.989995 899 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.482s (election complete) I0507 20:05:42.990123 899 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:43.270342 899 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:43.270484 899 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.870s (starting election) I0507 20:05:43.270617 899 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:43.271196 899 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:43.271373 899 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:43.271679 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.829s (SENT) I0507 20:05:43.271746 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:43.272006 899 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.428s (election complete) I0507 20:05:43.272094 899 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:43.272316 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:43.473103 899 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:43.473219 899 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.952s (starting election) I0507 20:05:43.473333 899 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:43.473772 899 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:43.473961 899 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:43.475021 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:43.701486 899 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:43.701656 899 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.540s (starting election) I0507 20:05:43.701768 899 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:43.702365 899 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:43.702591 899 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:43.703537 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:43.706324 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.817s (SENT) I0507 20:05:43.706398 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:43.706714 899 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.311s (election complete) I0507 20:05:43.706786 899 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:44.019071 899 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:44.019191 899 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.501s (starting election) I0507 20:05:44.019291 899 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:44.019767 899 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:44.019963 899 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:44.020736 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:44.143136 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.870s (SENT) I0507 20:05:44.143240 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:44.143607 899 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.650s (election complete) I0507 20:05:44.143712 899 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:44.244290 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.540s (SENT) I0507 20:05:44.244407 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:44.244726 899 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.687s (election complete) I0507 20:05:44.244827 899 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:44.426273 899 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:44.426424 899 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.410s (starting election) I0507 20:05:44.426512 899 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:44.427063 899 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:44.427284 899 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:44.427471 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.952s (SENT) I0507 20:05:44.427577 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:44.427834 899 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.839s (election complete) I0507 20:05:44.427922 899 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:44.428234 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:44.521479 899 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:05:44.521535 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.501s (SENT) I0507 20:05:44.521597 899 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.913s (starting election) I0507 20:05:44.521616 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:44.521708 899 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:44.522150 899 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:44.522316 899 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:44.522559 900 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.817s (election complete) I0507 20:05:44.522701 900 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:44.523188 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:44.839200 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.410s (SENT) I0507 20:05:44.839320 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:44.839753 900 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.981s (election complete) I0507 20:05:44.839875 900 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:44.932634 900 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:44.932787 900 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (starting election) I0507 20:05:44.932929 900 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:44.933590 900 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:44.933835 900 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:44.934615 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:45.035478 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:42.036153 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:05:42.036436 (+ 283us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:42.036450 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:42.036554 (+ 104us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:45.035174 (+2998620us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":92} W0507 20:05:45.035804 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:05:45.340591 900 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:45.340734 900 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.728s (starting election) I0507 20:05:45.340842 900 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:45.341292 900 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:45.341482 900 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:45.342583 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:45.382769 900 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:45.382931 900 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.897s (starting election) I0507 20:05:45.383096 900 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:45.383443 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.449s (SENT) I0507 20:05:45.383514 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:45.383690 900 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:45.383960 900 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:45.384212 903 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (election complete) I0507 20:05:45.384312 903 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:45.384949 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:45.437276 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.913s (SENT) I0507 20:05:45.437371 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:45.437762 903 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.344s (election complete) I0507 20:05:45.437878 903 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:45.782979 903 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:45.783146 903 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.884s (starting election) I0507 20:05:45.783262 903 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:45.783768 903 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:45.783980 903 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:45.785068 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:45.822104 903 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:45.822305 903 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.474s (starting election) I0507 20:05:45.822415 903 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:45.822983 903 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:45.823201 903 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:45.824184 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:46.071825 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.728s (SENT) I0507 20:05:46.071933 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:46.072466 903 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.497s (election complete) I0507 20:05:46.072655 903 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:46.101210 903 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:46.101392 903 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.427s (starting election) I0507 20:05:46.101549 903 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:46.102268 903 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:46.102587 903 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:46.103546 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:46.282641 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.897s (SENT) I0507 20:05:46.282739 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:46.283100 903 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.493s (election complete) I0507 20:05:46.283229 903 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:46.296939 903 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:46.297123 903 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.765s (starting election) I0507 20:05:46.297227 903 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:46.297706 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.474s (SENT) I0507 20:05:46.297797 903 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:46.297812 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:46.298061 903 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:46.298250 903 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.325s (election complete) I0507 20:05:46.298307 903 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:46.299283 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:46.530827 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.427s (SENT) I0507 20:05:46.530953 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:46.531417 904 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.093s (election complete) I0507 20:05:46.531594 904 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:46.570129 904 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:46.570305 904 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.677s (starting election) I0507 20:05:46.570444 904 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:46.571156 904 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:46.571444 904 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:46.572423 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:46.624414 904 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:46.624584 904 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.702s (starting election) I0507 20:05:46.624691 904 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:46.625237 904 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:46.625448 904 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:46.625706 903 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:46.625810 903 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.535s (starting election) I0507 20:05:46.625939 903 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:46.626291 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:46.626358 903 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:46.626503 903 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:46.626976 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:46.669374 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.884s (SENT) I0507 20:05:46.669481 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:46.669808 903 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.168s (election complete) I0507 20:05:46.669919 903 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:46.838249 903 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:46.838371 903 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.131s (starting election) I0507 20:05:46.838456 903 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:46.838901 903 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:46.839052 903 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:46.839921 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:46.970746 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.131s (SENT) I0507 20:05:46.970746 903 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:46.970901 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:46.970904 903 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.521s (starting election) I0507 20:05:46.971011 903 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:46.971437 903 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:46.971536 904 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.216s (election complete) I0507 20:05:46.971644 903 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:46.971662 904 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:46.972538 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:47.064373 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.765s (SENT) I0507 20:05:47.064465 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:47.064865 904 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.927s (election complete) I0507 20:05:47.064976 904 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:47.161619 904 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:47.161758 904 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.412s (starting election) I0507 20:05:47.161869 904 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:47.162405 904 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:47.162642 904 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:47.163214 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.535s (SENT) I0507 20:05:47.163321 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:05:47.163650 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:47.163884 904 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.198s (election complete) I0507 20:05:47.164059 904 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:47.188657 904 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:47.188817 904 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.090s (starting election) I0507 20:05:47.188907 904 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:47.189368 904 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:47.189563 904 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:47.190518 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:47.249027 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.677s (SENT) I0507 20:05:47.249121 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:47.249384 904 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.230s (election complete) I0507 20:05:47.249480 904 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:47.280529 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.090s (SENT) I0507 20:05:47.280686 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:47.281100 904 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.373s (election complete) I0507 20:05:47.281278 904 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:47.328841 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.702s (SENT) I0507 20:05:47.329097 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:47.329622 904 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.260s (election complete) I0507 20:05:47.329740 904 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:47.363137 904 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:47.363241 904 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.642s (starting election) I0507 20:05:47.363339 904 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:47.363735 904 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:47.363890 904 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:47.364755 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:47.494110 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.521s (SENT) I0507 20:05:47.494276 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:47.494801 904 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.497s (election complete) I0507 20:05:47.494951 904 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:47.575623 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.412s (SENT) I0507 20:05:47.575858 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:47.576342 904 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.391s (election complete) I0507 20:05:47.576506 904 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:47.590790 904 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:47.590972 904 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.284s (starting election) I0507 20:05:47.591102 904 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:47.591547 904 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:47.591698 904 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:47.592618 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:47.875898 904 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:47.876008 904 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.963s (starting election) I0507 20:05:47.876114 904 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:47.876487 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.284s (SENT) I0507 20:05:47.876559 904 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:47.876598 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:47.876778 904 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:47.876907 904 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.898s (election complete) I0507 20:05:47.876988 904 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:47.877809 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:47.968631 905 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:47.968747 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.633s (starting election) I0507 20:05:47.968924 905 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:47.969398 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:47.969610 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:47.970594 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:47.993048 905 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:47.993185 905 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.991s (starting election) I0507 20:05:47.993274 905 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:47.993690 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:47.993868 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:47.994957 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:48.006784 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.642s (SENT) I0507 20:05:48.006916 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:48.007290 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.304s (election complete) I0507 20:05:48.007398 905 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:48.075639 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:45.075650 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:05:45.075918 (+ 268us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:45.075931 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:45.076061 (+ 130us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:48.075317 (+2999256us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":86} W0507 20:05:48.075965 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:05:48.312860 905 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:48.313001 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.243s (starting election) I0507 20:05:48.313205 905 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:48.313726 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:48.313952 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:48.314785 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:48.557520 905 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:05:48.557657 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.243s (SENT) I0507 20:05:48.557711 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.690s (starting election) I0507 20:05:48.557826 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:48.557853 905 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:48.558399 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:48.558542 908 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.224s (election complete) I0507 20:05:48.558596 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:48.558620 908 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:48.559504 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:48.604522 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.633s (SENT) I0507 20:05:48.604634 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:48.605036 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.579s (election complete) I0507 20:05:48.605139 905 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:48.775985 905 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:48.776101 905 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.719s (starting election) I0507 20:05:48.776204 905 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:48.776643 905 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:48.776887 905 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:48.777941 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:48.841385 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.963s (SENT) I0507 20:05:48.841502 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:48.841876 905 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.216s (election complete) I0507 20:05:48.841994 905 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:48.984683 905 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:48.984827 905 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.376s (starting election) I0507 20:05:48.984931 905 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:48.985376 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:48.985560 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:48.985936 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.991s (SENT) I0507 20:05:48.986032 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:48.986366 905 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.340s (election complete) I0507 20:05:48.986492 905 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:48.986642 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:49.059325 905 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:49.059442 905 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.980s (starting election) I0507 20:05:49.059548 905 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:49.059972 905 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:49.060164 905 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:49.061115 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:49.184823 905 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:49.185012 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.420s (starting election) I0507 20:05:49.185122 905 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:49.185636 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:49.185849 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:49.186714 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:49.249899 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.690s (SENT) I0507 20:05:49.250005 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:49.250399 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.843s (election complete) I0507 20:05:49.250509 905 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:49.326813 905 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:49.326998 905 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.928s (starting election) I0507 20:05:49.327121 905 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:49.327764 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:49.328032 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:49.329011 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:49.361795 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.376s (SENT) I0507 20:05:49.361893 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:49.362272 905 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.555s (election complete) I0507 20:05:49.362402 905 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:49.498029 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.719s (SENT) I0507 20:05:49.498132 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:49.498497 905 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.950s (election complete) I0507 20:05:49.498630 905 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:49.606150 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.420s (SENT) I0507 20:05:49.606245 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:49.606590 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.516s (election complete) I0507 20:05:49.606719 905 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:49.917917 905 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:49.918078 905 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.208s (starting election) I0507 20:05:49.918181 905 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:49.918684 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:49.918898 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:49.920295 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:50.040843 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.980s (SENT) I0507 20:05:50.040971 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:50.041405 905 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.162s (election complete) I0507 20:05:50.041539 905 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:50.123540 905 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:50.123700 905 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.670s (starting election) I0507 20:05:50.123831 905 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:50.124331 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:50.124552 905 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:50.125572 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:50.127490 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.208s (SENT) I0507 20:05:50.127507 905 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:50.127573 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:50.127671 905 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.483s (starting election) I0507 20:05:50.127766 905 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:50.128290 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:50.128363 910 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.238s (election complete) I0507 20:05:50.128476 905 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:50.128458 910 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:50.129343 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:50.204866 910 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:50.205047 910 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.950s (starting election) I0507 20:05:50.205195 910 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:50.205986 910 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:50.206284 910 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:50.207392 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:50.257505 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.928s (SENT) I0507 20:05:50.257623 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:50.258029 910 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.256s (election complete) I0507 20:05:50.258154 910 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:50.515380 910 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:50.515554 910 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.848s (starting election) I0507 20:05:50.515660 910 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:50.516178 910 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:50.516382 910 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:50.517315 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:50.612365 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.483s (SENT) I0507 20:05:50.612494 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:50.612887 910 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.881s (election complete) I0507 20:05:50.612974 910 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:50.795290 910 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:50.795472 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.471s (starting election) I0507 20:05:50.795624 910 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:50.796293 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:50.796679 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.670s (SENT) I0507 20:05:50.796726 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:50.796771 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:50.797019 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.206s (election complete) I0507 20:05:50.797139 910 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:50.797686 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:51.004328 910 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:51.004501 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.080s (starting election) I0507 20:05:51.004623 910 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:51.005321 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:51.005568 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:51.006410 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:51.085453 910 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:51.085590 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.663s (starting election) I0507 20:05:51.085705 910 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:51.086264 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:51.086474 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:51.086771 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.080s (SENT) I0507 20:05:51.086846 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:51.087162 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.420s (election complete) I0507 20:05:51.087280 910 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:51.087488 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:51.119457 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:48.121909 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:05:48.122221 (+ 312us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:48.122235 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:48.122349 (+ 114us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:51.119173 (+2996824us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":74} W0507 20:05:51.119758 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:05:51.156344 910 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:51.156471 910 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.185s (starting election) I0507 20:05:51.156561 910 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:51.157028 910 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:51.157207 910 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:51.157315 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.950s (SENT) I0507 20:05:51.157378 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:51.157598 910 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.402s (election complete) I0507 20:05:51.157655 910 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:51.158123 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:51.268669 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.471s (SENT) I0507 20:05:51.268772 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:51.269086 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.723s (election complete) I0507 20:05:51.269161 910 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:51.343353 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.185s (SENT) I0507 20:05:51.343447 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:51.343758 910 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (election complete) I0507 20:05:51.343838 910 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:51.366076 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.848s (SENT) I0507 20:05:51.366175 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:51.366468 910 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.459s (election complete) I0507 20:05:51.366572 910 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:51.665993 910 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:51.666198 910 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.527s (starting election) I0507 20:05:51.666355 910 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:51.667070 910 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:51.667356 910 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:51.668676 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:51.751081 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.663s (SENT) I0507 20:05:51.751188 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:51.751582 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.273s (election complete) I0507 20:05:51.751771 910 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:51.826217 910 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:51.826397 910 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.809s (starting election) I0507 20:05:51.826515 910 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:51.827108 910 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:51.827307 910 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:51.828267 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:52.026334 910 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:52.026520 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.439s (starting election) I0507 20:05:52.026679 910 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:52.027482 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:52.027777 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:52.028766 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:52.193878 910 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:52.194105 910 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.314s (starting election) I0507 20:05:52.194248 910 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:52.195169 910 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:52.195446 910 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:52.195909 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.527s (SENT) I0507 20:05:52.196010 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:52.196264 910 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.588s (election complete) I0507 20:05:52.196347 910 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:52.196482 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:52.467021 910 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:52.467211 910 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.375s (starting election) I0507 20:05:52.467365 910 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:52.467716 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.439s (SENT) I0507 20:05:52.467787 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:52.468076 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:52.468309 910 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:52.468371 914 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (election complete) I0507 20:05:52.468436 914 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:52.469220 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:52.510080 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.314s (SENT) I0507 20:05:52.510206 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:52.510620 914 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.806s (election complete) I0507 20:05:52.510716 914 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:52.569352 914 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:52.569501 914 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.445s (starting election) I0507 20:05:52.569622 914 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:52.570165 914 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:52.570391 914 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:52.571305 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:52.636512 914 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:52.636706 914 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (starting election) I0507 20:05:52.636806 914 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:52.637290 914 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:52.637465 914 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:52.637753 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.809s (SENT) I0507 20:05:52.637825 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:52.638250 914 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.488s (election complete) I0507 20:05:52.638339 914 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:52.638597 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:52.844133 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.375s (SENT) I0507 20:05:52.844274 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:52.844765 914 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.076s (election complete) I0507 20:05:52.844880 914 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:52.921536 914 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:52.921648 914 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.796s (starting election) I0507 20:05:52.921737 914 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:52.922173 914 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:52.922482 914 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:52.923290 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:53.015858 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.445s (SENT) I0507 20:05:53.016021 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:53.016515 914 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.931s (election complete) I0507 20:05:53.016671 914 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:53.087286 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.449s (SENT) I0507 20:05:53.087415 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:53.087798 914 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.570s (election complete) I0507 20:05:53.087945 914 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:53.317765 914 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:53.317939 914 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.693s (starting election) I0507 20:05:53.318040 914 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:53.318519 914 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:53.318701 914 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:53.319738 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:53.658262 914 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:53.658404 914 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.329s (starting election) I0507 20:05:53.658506 914 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:53.659025 914 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:53.659219 914 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:53.660202 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:53.720846 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.796s (SENT) I0507 20:05:53.720999 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:53.721431 914 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.688s (election complete) I0507 20:05:53.721585 914 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:53.987903 914 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:53.988056 914 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.227s (starting election) I0507 20:05:53.988142 914 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:53.988651 914 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:53.988867 914 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:53.989127 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.329s (SENT) I0507 20:05:53.989219 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:53.989531 914 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.868s (election complete) I0507 20:05:53.989608 914 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:53.989801 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:54.012219 914 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:54.012509 914 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.711s (starting election) I0507 20:05:54.012696 914 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:54.013180 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.693s (SENT) I0507 20:05:54.013250 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:54.013406 914 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:54.013721 914 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:54.013844 915 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.891s (election complete) I0507 20:05:54.013965 915 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:54.014648 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:54.175576 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:51.177282 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:05:51.177542 (+ 260us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:51.177557 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:51.177671 (+ 114us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:54.175225 (+2997554us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":69} W0507 20:05:54.175932 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:05:54.216787 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.227s (SENT) I0507 20:05:54.216922 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:54.217314 915 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.724s (election complete) I0507 20:05:54.217427 915 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:54.410776 915 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:54.410959 915 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.494s (starting election) I0507 20:05:54.411103 915 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:54.411917 915 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:54.412173 915 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:54.413139 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:54.726034 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.711s (SENT) I0507 20:05:54.726162 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:54.726544 915 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.980s (election complete) I0507 20:05:54.726680 915 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:54.906304 915 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:54.906497 915 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.365s (starting election) I0507 20:05:54.906628 915 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:54.907291 915 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:54.907400 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.494s (SENT) I0507 20:05:54.907480 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:54.907603 915 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:54.907809 915 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.155s (election complete) I0507 20:05:54.907980 915 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:54.908519 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:54.942478 918 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:54.942663 918 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.247s (starting election) I0507 20:05:54.942801 918 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:54.943609 918 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:54.943895 918 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:54.945058 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:55.064136 918 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:55.064347 918 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.314s (starting election) I0507 20:05:55.064482 918 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:55.065172 918 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:55.065435 918 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:55.066351 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:55.190273 918 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:55.190443 918 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.678s (starting election) I0507 20:05:55.190601 918 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:55.191354 918 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:55.191359 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.247s (SENT) I0507 20:05:55.191460 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:55.191674 918 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:55.191738 915 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.542s (election complete) I0507 20:05:55.191805 915 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:55.192950 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:55.274266 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.365s (SENT) I0507 20:05:55.274384 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:55.274770 915 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.643s (election complete) I0507 20:05:55.274865 915 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:55.380213 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.314s (SENT) I0507 20:05:55.380372 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:55.380837 915 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.710s (election complete) I0507 20:05:55.380934 915 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:55.707976 915 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:55.708109 915 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.477s (starting election) I0507 20:05:55.708230 915 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:55.708765 915 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:55.708976 915 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:55.711139 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:55.734472 915 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:55.734624 915 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.446s (starting election) I0507 20:05:55.734725 915 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:55.735285 915 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:55.735446 915 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:55.736472 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:55.810243 915 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:55.810400 915 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.428s (starting election) I0507 20:05:55.810554 915 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:55.811185 915 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:55.811460 915 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:55.812629 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:55.870796 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.678s (SENT) I0507 20:05:55.870939 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:55.871292 915 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.095s (election complete) I0507 20:05:55.871402 915 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:55.967813 915 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:55.967965 915 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.970s (starting election) I0507 20:05:55.968075 915 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:55.968555 915 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:55.968750 915 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:55.969817 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:56.092134 915 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:56.092283 915 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.708s (starting election) I0507 20:05:56.092383 915 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:56.092839 915 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:56.093042 915 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:56.094023 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:56.183091 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.446s (SENT) I0507 20:05:56.183266 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:56.183683 915 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.424s (election complete) I0507 20:05:56.183831 915 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:56.185642 915 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:56.185760 915 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.816s (starting election) I0507 20:05:56.185850 915 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:56.186466 915 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:56.186643 915 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:56.186982 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.477s (SENT) I0507 20:05:56.187059 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:56.187280 915 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.998s (election complete) I0507 20:05:56.187338 915 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:56.187574 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:56.240267 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.428s (SENT) I0507 20:05:56.240401 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:56.240778 915 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.908s (election complete) I0507 20:05:56.240882 915 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:56.801259 920 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:56.801409 920 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.626s (starting election) I0507 20:05:56.801543 920 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:56.802151 920 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:56.802305 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.708s (SENT) I0507 20:05:56.802390 920 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:56.802434 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:56.802692 920 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.954s (election complete) I0507 20:05:56.802785 920 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:56.803876 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:56.941200 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.970s (SENT) I0507 20:05:56.941351 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:56.941789 920 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.786s (election complete) I0507 20:05:56.941881 920 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:57.004071 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.816s (SENT) I0507 20:05:57.004199 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:57.004561 920 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.463s (election complete) I0507 20:05:57.004673 920 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:57.247815 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:54.247167 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:05:54.247531 (+ 364us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:54.247564 (+ 33us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:54.247738 (+ 174us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:05:57.247433 (+2999695us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":108} W0507 20:05:57.248266 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:05:57.429644 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.626s (SENT) I0507 20:05:57.429770 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:57.430122 920 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.377s (election complete) I0507 20:05:57.430233 920 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:57.468170 920 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:57.468323 920 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.206s (starting election) I0507 20:05:57.468420 920 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:57.468894 920 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:57.469105 920 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:57.470419 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:57.674772 920 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:57.674927 920 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.291s (starting election) I0507 20:05:57.675047 920 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:05:57.675495 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.206s (SENT) I0507 20:05:57.675595 920 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:57.675596 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:57.675803 920 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:57.675945 920 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.348s (election complete) I0507 20:05:57.676087 920 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:57.676955 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:57.728523 925 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:57.728725 925 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.515s (starting election) I0507 20:05:57.728829 925 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:57.729426 925 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:57.729696 925 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:57.731142 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:57.808228 925 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:57.808389 925 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.806s (starting election) I0507 20:05:57.808485 925 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:57.808954 925 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:57.809176 925 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:57.810137 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:57.968286 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.291s (SENT) I0507 20:05:57.968399 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:57.968830 925 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.705s (election complete) I0507 20:05:57.968946 925 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:58.244390 925 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:58.244532 925 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.181s (starting election) I0507 20:05:58.244650 925 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:58.245149 925 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:58.245357 925 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:58.246486 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:58.246909 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.515s (SENT) I0507 20:05:58.246969 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:58.247220 925 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.686s (election complete) I0507 20:05:58.247335 925 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:58.427625 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.181s (SENT) I0507 20:05:58.427765 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:58.428171 925 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.597s (election complete) I0507 20:05:58.428303 925 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:58.614945 925 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:58.615162 925 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.175s (starting election) I0507 20:05:58.615363 925 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:58.616057 925 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:58.616236 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.806s (SENT) I0507 20:05:58.616331 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:58.616360 925 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:58.616633 925 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.443s (election complete) I0507 20:05:58.616734 925 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:58.617270 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:58.675143 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:58.675369 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.493s (starting election) I0507 20:05:58.675521 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:58.676151 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:58.676445 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:58.677340 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:58.793380 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.175s (SENT) I0507 20:05:58.793521 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:58.793992 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.949s (election complete) I0507 20:05:58.794152 926 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:05:59.026439 926 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:59.026616 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.433s (starting election) I0507 20:05:59.026733 926 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:59.027436 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:59.027660 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:59.028687 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:59.169184 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:59.169378 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.308s (starting election) I0507 20:05:59.169528 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:59.170104 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:59.170361 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:59.170734 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.493s (SENT) I0507 20:05:59.170809 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:59.171058 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.355s (election complete) I0507 20:05:59.171178 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:05:59.171376 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:59.460705 926 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:59.460901 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.991s (starting election) I0507 20:05:59.461035 926 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:59.461746 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:59.461796 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.433s (SENT) I0507 20:05:59.461869 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:05:59.461978 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:59.462247 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.518s (election complete) I0507 20:05:59.462383 926 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:05:59.463133 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:05:59.479796 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.308s (SENT) I0507 20:05:59.479918 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:05:59.480303 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.675s (election complete) I0507 20:05:59.480417 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:05:59.744050 926 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:59.744221 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (starting election) I0507 20:05:59.744344 926 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:05:59.744925 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:05:59.745170 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:05:59.746098 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:05:59.981752 926 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:05:59.981937 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.604s (starting election) I0507 20:05:59.982097 926 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:05:59.982678 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:05:59.982934 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:05:59.984050 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:00.156724 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:00.156852 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.613s (starting election) I0507 20:06:00.156980 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:00.157431 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:00.157634 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:00.158562 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:00.323621 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:05:57.325201 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:05:57.325483 (+ 282us) negotiation.cc:98] Waiting for socket to connect 0507 20:05:57.325497 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:05:57.325608 (+ 111us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:00.323297 (+2997689us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":84} W0507 20:06:00.323900 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:06:00.454602 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.991s (SENT) I0507 20:06:00.454741 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:00.455159 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.901s (election complete) I0507 20:06:00.455327 926 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:00.587738 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.604s (SENT) I0507 20:06:00.587877 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:00.588326 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.194s (election complete) I0507 20:06:00.588479 926 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:00.608397 926 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:00.608570 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.638s (starting election) I0507 20:06:00.608688 926 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:00.609319 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:00.609560 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:00.610069 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.863s (SENT) I0507 20:06:00.610188 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:06:00.610435 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:00.610476 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.095s (election complete) I0507 20:06:00.610543 926 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:00.706393 926 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:00.706543 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.751s (starting election) I0507 20:06:00.706692 926 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:00.707314 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:00.707617 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:00.708519 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:00.770460 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:00.770613 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.167s (starting election) I0507 20:06:00.770716 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:00.771244 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:00.771447 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:00.771755 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.613s (SENT) I0507 20:06:00.771821 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:00.772147 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.146s (election complete) I0507 20:06:00.772259 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:00.772519 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:00.783116 926 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:00.783257 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.076s (starting election) I0507 20:06:00.783370 926 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:00.783910 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:00.784168 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:00.785049 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:00.860903 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.076s (SENT) I0507 20:06:00.860842 926 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:00.861055 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (starting election) I0507 20:06:00.861140 926 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:00.861582 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:00.861733 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:00.861925 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:00.862089 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.771s (election complete) I0507 20:06:00.862138 926 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:00.862856 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:00.919679 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:00.919842 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.793s (starting election) I0507 20:06:00.919981 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:00.920626 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:00.920908 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:00.921876 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:00.939781 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.167s (SENT) I0507 20:06:00.939903 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:00.940377 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.261s (election complete) I0507 20:06:00.940583 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:01.202850 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:01.203073 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.205s (starting election) I0507 20:06:01.203202 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:01.203869 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:01.204073 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:01.204845 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:01.249835 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.638s (SENT) I0507 20:06:01.249981 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:01.250439 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.283s (election complete) I0507 20:06:01.250545 926 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:01.408923 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:01.409164 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.547s (starting election) I0507 20:06:01.409291 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:01.409917 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:01.410169 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:01.410303 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.205s (SENT) I0507 20:06:01.410421 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:01.410771 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.224s (election complete) I0507 20:06:01.410866 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:01.410995 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:01.459578 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.751s (SENT) I0507 20:06:01.459705 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:01.460096 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.169s (election complete) I0507 20:06:01.460249 926 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:01.579180 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.716s (SENT) I0507 20:06:01.579322 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:01.579785 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (election complete) I0507 20:06:01.579943 926 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:01.630175 926 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:01.630308 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.353s (starting election) I0507 20:06:01.630421 926 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:01.630945 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:01.631117 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:01.631944 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:01.636112 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:01.636206 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.818s (starting election) I0507 20:06:01.636292 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:01.636667 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:01.636850 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:01.637470 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:01.715562 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.793s (SENT) I0507 20:06:01.715659 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:01.715968 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.536s (election complete) I0507 20:06:01.716068 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:01.959139 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.547s (SENT) I0507 20:06:01.959242 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:01.959561 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.212s (election complete) I0507 20:06:01.959674 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:01.984261 926 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:01.984411 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.092s (starting election) I0507 20:06:01.984540 926 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:01.984685 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.353s (SENT) I0507 20:06:01.984764 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:01.985018 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:01.985513 930 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.707s (election complete) I0507 20:06:01.985671 930 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:01.985744 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:01.986182 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:02.030009 926 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:02.030186 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.240s (starting election) I0507 20:06:02.030354 926 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:02.031054 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:02.031343 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:02.032456 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:02.079236 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.092s (SENT) I0507 20:06:02.079375 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:02.079880 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.990s (election complete) I0507 20:06:02.080040 926 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:02.172660 926 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:02.172874 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.588s (starting election) I0507 20:06:02.173000 926 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:02.173707 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:02.173960 926 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:02.174926 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:02.270839 926 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:02.271015 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.852s (starting election) I0507 20:06:02.271138 926 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:02.271657 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:02.271849 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:02.272132 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.240s (SENT) I0507 20:06:02.272313 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:02.272830 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.598s (election complete) W0507 20:06:02.273080 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:02.273097 926 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:02.456493 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.818s (SENT) I0507 20:06:02.456598 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:02.456969 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.966s (election complete) I0507 20:06:02.457069 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:02.764055 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.588s (SENT) I0507 20:06:02.764159 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:02.764533 926 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.950s (election complete) I0507 20:06:02.764631 926 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:02.871948 926 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:02.872092 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.271s (starting election) I0507 20:06:02.872195 926 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:02.872714 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:02.872941 926 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:02.874245 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:03.070996 926 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:03.071144 926 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.596s (starting election) I0507 20:06:03.071274 926 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:03.071851 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:03.072088 926 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:03.073022 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:03.125582 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.852s (SENT) I0507 20:06:03.125710 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:03.126271 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.395s (election complete) I0507 20:06:03.126448 926 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:03.144632 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.271s (SENT) I0507 20:06:03.144749 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:03.145108 926 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.819s (election complete) I0507 20:06:03.145241 926 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:03.411514 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:00.410938 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:00.411261 (+ 323us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:00.411284 (+ 23us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:00.411425 (+ 141us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:03.411185 (+2999760us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":93} W0507 20:06:03.411759 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:03.668632 934 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:03.668733 934 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.284s (starting election) I0507 20:06:03.668833 934 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:03.669229 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.596s (SENT) I0507 20:06:03.669312 934 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:03.669373 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:03.669543 934 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:03.669792 934 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.676s (election complete) I0507 20:06:03.669988 934 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:03.670665 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:03.716089 935 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:03.716275 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.885s (starting election) I0507 20:06:03.716434 935 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:03.717094 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:03.717429 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:03.718437 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:03.955590 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.284s (SENT) I0507 20:06:03.955742 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:03.956130 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.497s (election complete) I0507 20:06:03.956241 935 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:03.965353 935 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:03.965533 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.232s (starting election) I0507 20:06:03.965641 935 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:03.966150 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:03.966341 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:03.967357 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:04.198268 935 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:04.198406 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.252s (starting election) I0507 20:06:04.198532 935 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:04.199044 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:04.199260 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:04.199496 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.232s (SENT) I0507 20:06:04.199574 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:04.199846 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.251s (election complete) I0507 20:06:04.199949 935 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:04.200309 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:04.452168 935 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:04.452252 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.252s (SENT) I0507 20:06:04.452339 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:04.452338 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.944s (starting election) I0507 20:06:04.452445 935 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:04.452996 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:04.453058 937 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.979s (election complete) I0507 20:06:04.453152 937 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:04.453202 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:04.453490 935 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:04.453586 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.448s (starting election) I0507 20:06:04.453670 935 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:04.454107 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:04.454237 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:04.454273 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:04.454839 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:04.602358 935 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:04.602517 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.436s (starting election) I0507 20:06:04.602643 935 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:04.603268 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:04.603474 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:04.603811 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.885s (SENT) I0507 20:06:04.603895 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:04.604265 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.655s (election complete) I0507 20:06:04.604426 935 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:04.604429 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:04.902146 935 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:04.902272 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.301s (starting election) I0507 20:06:04.902385 935 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:04.902854 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:04.903084 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:04.903163 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.448s (SENT) I0507 20:06:04.903260 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:04.903492 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.949s (election complete) I0507 20:06:04.903591 935 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:04.904093 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:05.040622 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.436s (SENT) I0507 20:06:05.040740 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:05.041163 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (election complete) I0507 20:06:05.041303 935 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:05.140923 935 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:05.141070 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.481s (starting election) I0507 20:06:05.141170 935 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:05.141682 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:05.141870 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:05.143010 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:05.205113 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.301s (SENT) I0507 20:06:05.205245 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:05.205569 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.336s (election complete) I0507 20:06:05.205660 935 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:05.398844 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.944s (SENT) I0507 20:06:05.398972 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:05.399399 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.115s (election complete) I0507 20:06:05.399520 935 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:05.515401 935 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:05.515542 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.949s (starting election) I0507 20:06:05.515660 935 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:05.516151 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:05.516383 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:05.517421 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:05.542737 935 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:05.542929 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.431s (starting election) I0507 20:06:05.543057 935 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:05.543562 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:05.543772 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:05.544806 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:05.623661 935 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:05.623831 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.676s (starting election) W0507 20:06:05.623836 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.481s (SENT) I0507 20:06:05.623935 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:05.623980 935 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:05.624712 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:05.624708 938 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.728s (election complete) I0507 20:06:05.624794 938 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:05.624969 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:05.625924 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:05.974418 935 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:05.974593 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.443s (starting election) I0507 20:06:05.974750 935 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:05.975297 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:05.975517 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:05.975692 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.431s (SENT) I0507 20:06:05.975787 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:05.976070 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.988s (election complete) I0507 20:06:05.976142 935 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:05.976627 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:06.301937 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.676s (SENT) I0507 20:06:06.302039 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:06.302418 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.284s (election complete) I0507 20:06:06.302527 935 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:06.419986 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.443s (SENT) I0507 20:06:06.420087 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:06.420534 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.310s (election complete) I0507 20:06:06.420655 935 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:06.464995 935 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:06.465108 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.222s (starting election) I0507 20:06:06.465212 935 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:06.465664 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:06.465837 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:06.466089 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.949s (SENT) I0507 20:06:06.466176 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:06.466465 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.416s (election complete) I0507 20:06:06.466619 935 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:06.466686 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:06.507534 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:03.507840 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:03.508207 (+ 367us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:03.508228 (+ 21us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:03.508343 (+ 115us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:06.507218 (+2998875us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":101} W0507 20:06:06.507885 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:06.587178 935 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:06.587333 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.770s (starting election) I0507 20:06:06.587432 935 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:06.587885 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:06.588068 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:06.589064 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:06.688921 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.222s (SENT) I0507 20:06:06.689051 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:06.689535 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.183s (election complete) I0507 20:06:06.689656 935 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:06.732008 935 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:06.732120 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.563s (starting election) I0507 20:06:06.732218 935 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:06.732677 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:06.732867 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:06.733747 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:06.873261 935 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:06.873397 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.736s (starting election) I0507 20:06:06.873499 935 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:06.873950 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:06.874083 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:06.875054 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:07.295812 935 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:07.295925 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.312s (starting election) I0507 20:06:07.296020 935 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:07.296525 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:07.296695 935 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:07.297436 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:07.298223 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.563s (SENT) I0507 20:06:07.298290 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:07.298604 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.206s (election complete) I0507 20:06:07.298691 935 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:07.358707 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.770s (SENT) I0507 20:06:07.358827 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:07.358814 935 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:07.358917 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.158s (starting election) I0507 20:06:07.359009 935 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:07.359486 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:07.359669 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:07.359807 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.191s (election complete) I0507 20:06:07.359858 935 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:07.360463 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:07.505712 941 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:07.505831 941 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.207s (starting election) I0507 20:06:07.505924 941 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:07.506381 941 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:07.506610 941 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:07.507411 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:07.518613 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.158s (SENT) I0507 20:06:07.518724 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:07.519055 941 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.295s (election complete) I0507 20:06:07.519165 941 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:07.610031 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.312s (SENT) I0507 20:06:07.610129 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:07.610468 941 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.980s (election complete) I0507 20:06:07.610611 941 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:07.610618 935 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:07.610677 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.869s (starting election) I0507 20:06:07.610769 935 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:07.611176 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:07.611342 935 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:07.611544 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.736s (SENT) I0507 20:06:07.611603 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:07.611737 935 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.965s (election complete) I0507 20:06:07.611801 935 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:07.612360 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:07.714937 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.207s (SENT) I0507 20:06:07.715034 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:07.715340 935 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (election complete) I0507 20:06:07.715445 935 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:07.815361 935 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:07.815488 935 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.987s (starting election) I0507 20:06:07.815582 935 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:07.816026 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:07.816187 935 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:07.816943 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:08.481250 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.869s (SENT) I0507 20:06:08.481359 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:08.481896 942 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.331s (election complete) I0507 20:06:08.481966 942 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:08.579813 942 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:08.579936 942 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.418s (starting election) I0507 20:06:08.580041 942 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:08.580548 942 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:08.580732 942 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:08.581910 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:08.803833 942 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:08.803951 942 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.126s (starting election) I0507 20:06:08.804056 942 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:08.804293 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.987s (SENT) I0507 20:06:08.804376 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:08.804477 942 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:08.804630 942 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:08.804783 942 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.372s (election complete) I0507 20:06:08.804924 942 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:08.805209 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:08.813567 944 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:08.813709 944 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.989s (starting election) I0507 20:06:08.813828 944 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:08.814363 944 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:08.814601 944 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:08.815527 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:08.931818 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.126s (SENT) I0507 20:06:08.931917 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:08.932325 944 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.858s (election complete) I0507 20:06:08.932474 944 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:08.998638 944 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:08.998807 944 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.911s (starting election) I0507 20:06:08.998937 944 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:08.999430 944 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:08.999649 944 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:09.000010 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.418s (SENT) I0507 20:06:09.000141 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:06:09.000450 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:09.000468 944 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.811s (election complete) I0507 20:06:09.000556 944 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:09.607476 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:06.607811 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:06.608155 (+ 344us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:06.608172 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:06.608295 (+ 123us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:09.607181 (+2998886us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":105} W0507 20:06:09.607738 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:09.791677 947 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:09.791811 947 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.453s (starting election) I0507 20:06:09.791930 947 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:09.792469 947 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:09.792702 947 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:09.793872 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:09.803244 947 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:09.803416 947 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.931s (starting election) I0507 20:06:09.803552 947 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:09.804060 947 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:09.804267 947 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:09.805198 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:09.805941 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.989s (SENT) I0507 20:06:09.806038 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:09.806288 947 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.775s (election complete) I0507 20:06:09.806382 947 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:09.812729 947 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:09.812839 947 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.300s (starting election) I0507 20:06:09.812959 947 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:09.813431 947 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:09.813684 947 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:09.814532 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:09.911957 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.911s (SENT) I0507 20:06:09.912094 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:09.912554 947 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.481s (election complete) I0507 20:06:09.912693 947 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:10.114955 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.300s (SENT) I0507 20:06:10.115082 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:10.115521 947 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.752s (election complete) I0507 20:06:10.115649 947 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:10.246242 947 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:10.246376 947 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.924s (starting election) I0507 20:06:10.246477 947 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:10.246992 947 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:10.247086 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.453s (SENT) I0507 20:06:10.247267 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:10.247273 947 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:10.247570 947 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.993s (election complete) I0507 20:06:10.247632 947 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:10.248001 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:10.582638 947 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:10.582756 947 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.951s (starting election) I0507 20:06:10.582904 947 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:10.583392 947 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:10.583596 947 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:10.584473 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:10.736958 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.931s (SENT) I0507 20:06:10.737066 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:10.737385 947 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.701s (election complete) I0507 20:06:10.737479 947 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:10.868966 947 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:10.869107 947 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.438s (starting election) I0507 20:06:10.869191 947 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:10.869627 947 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:10.869807 947 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:10.870595 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:11.172574 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.924s (SENT) I0507 20:06:11.172684 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:11.173014 947 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.448s (election complete) I0507 20:06:11.173092 947 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:11.307507 947 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:11.307626 947 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.356s (starting election) I0507 20:06:11.307725 947 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:11.308207 947 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:11.308272 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.438s (SENT) I0507 20:06:11.308344 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:11.308395 947 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:11.308499 947 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.595s (election complete) I0507 20:06:11.308567 947 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:11.309309 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:11.439105 949 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:11.439252 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.851s (starting election) I0507 20:06:11.439381 949 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:11.439920 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:11.440150 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:11.441148 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:11.536140 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.951s (SENT) I0507 20:06:11.536231 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:11.536592 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.272s (election complete) I0507 20:06:11.536725 949 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:11.622288 949 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:11.622476 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.824s (starting election) I0507 20:06:11.622588 949 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:11.623121 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:11.623318 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:11.624101 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:11.666014 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.356s (SENT) I0507 20:06:11.666129 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:11.666435 949 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.686s (election complete) I0507 20:06:11.666513 949 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:11.809409 949 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:11.809516 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.337s (starting election) I0507 20:06:11.809623 949 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:11.810012 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:11.810170 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:11.811287 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:12.147732 949 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:12.147816 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.337s (SENT) I0507 20:06:12.147859 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.661s (starting election) I0507 20:06:12.147893 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:12.147946 949 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:12.148423 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:12.148422 950 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.291s (election complete) I0507 20:06:12.148548 950 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:12.148603 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:12.149298 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:12.292358 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.851s (SENT) I0507 20:06:12.292467 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:12.292793 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.915s (election complete) I0507 20:06:12.292878 949 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:12.353461 949 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:12.353580 949 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.780s (starting election) I0507 20:06:12.353665 949 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:12.354137 949 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:12.354322 949 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:12.355028 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:12.447293 949 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:12.447435 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.695s (starting election) I0507 20:06:12.447527 949 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:12.447998 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:12.448217 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:12.448496 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.824s (SENT) I0507 20:06:12.448563 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:12.448909 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.609s (election complete) W0507 20:06:12.448947 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:12.449046 949 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:12.715469 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:09.715296 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:09.715646 (+ 350us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:09.715660 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:09.715787 (+ 127us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:12.715196 (+2999409us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":165} W0507 20:06:12.715706 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:06:12.811270 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.661s (SENT) I0507 20:06:12.811408 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:12.811902 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.341s (election complete) I0507 20:06:12.812018 949 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:13.058567 949 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:13.058673 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.627s (starting election) I0507 20:06:13.058768 949 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:13.059216 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:13.059394 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:13.060676 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:13.134115 949 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:13.134326 949 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.825s (starting election) I0507 20:06:13.134438 949 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:13.135015 949 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:13.135218 949 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:13.135608 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.780s (SENT) I0507 20:06:13.135721 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:13.135964 949 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.686s (election complete) I0507 20:06:13.136039 949 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:13.136037 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:13.144398 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.695s (SENT) I0507 20:06:13.144482 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:13.144768 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.861s (election complete) I0507 20:06:13.144857 949 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:13.153947 949 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:13.154100 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.285s (starting election) I0507 20:06:13.154222 949 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:13.154722 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:13.154945 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:13.157292 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:13.440259 949 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:13.440322 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.285s (SENT) I0507 20:06:13.440428 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:13.440433 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.728s (starting election) I0507 20:06:13.440536 949 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:13.441134 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:13.441215 954 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.728s (election complete) I0507 20:06:13.441356 954 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:13.441376 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:13.442385 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:13.688735 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.627s (SENT) I0507 20:06:13.688841 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:13.689188 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.458s (election complete) I0507 20:06:13.689280 949 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:13.822845 949 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:13.823016 949 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.941s (starting election) I0507 20:06:13.823149 949 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:13.823706 949 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:13.823920 949 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:13.824822 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:13.960958 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.825s (SENT) I0507 20:06:13.961055 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:13.961364 949 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.826s (election complete) I0507 20:06:13.961454 949 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:14.147640 949 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:14.147778 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.361s (starting election) I0507 20:06:14.147867 949 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:14.148326 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:14.148489 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:14.149564 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:14.169721 949 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:14.169847 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.219s (starting election) I0507 20:06:14.169952 949 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:14.170402 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:14.170589 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:14.170841 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.728s (SENT) I0507 20:06:14.170953 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:14.171159 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.192s (election complete) I0507 20:06:14.171229 949 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:14.171727 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:14.364605 949 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:14.364744 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.206s (starting election) I0507 20:06:14.364850 949 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:14.365344 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:14.365515 949 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:14.366601 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:14.390192 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.219s (SENT) I0507 20:06:14.390291 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:14.390580 949 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.488s (election complete) I0507 20:06:14.390655 949 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:14.509745 949 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:14.509793 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.361s (SENT) I0507 20:06:14.509932 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.930s (starting election) I0507 20:06:14.509965 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:14.510061 949 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:14.510607 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:14.510830 949 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:14.510985 949 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.466s (election complete) I0507 20:06:14.511049 949 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:14.512096 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:14.572430 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.206s (SENT) I0507 20:06:14.572547 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:14.572935 955 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.647s (election complete) I0507 20:06:14.573061 955 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:14.765982 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.941s (SENT) I0507 20:06:14.766085 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:14.766518 955 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.881s (election complete) I0507 20:06:14.766636 955 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:14.977843 955 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:14.978011 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.245s (starting election) I0507 20:06:14.978163 955 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:14.979872 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:14.980175 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:14.981226 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:15.220894 955 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:15.221076 955 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.227s (starting election) I0507 20:06:15.221197 955 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:15.221801 955 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:15.222033 955 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:15.223058 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:15.223783 955 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:15.223894 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.407s (starting election) I0507 20:06:15.224015 955 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:15.224438 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:15.224581 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:15.225363 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:15.226081 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.245s (SENT) I0507 20:06:15.226151 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:15.226341 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.163s (election complete) I0507 20:06:15.226423 955 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:15.390213 955 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:15.390384 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.348s (starting election) I0507 20:06:15.390496 955 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:15.391153 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:15.391386 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:15.392298 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:15.442905 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.930s (SENT) I0507 20:06:15.443008 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:15.443408 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.867s (election complete) I0507 20:06:15.443493 955 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:15.448892 955 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:15.449003 955 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.813s (starting election) I0507 20:06:15.449108 955 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:15.449571 955 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:15.449803 955 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:15.450191 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.227s (SENT) I0507 20:06:15.450307 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:15.450659 955 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.188s (election complete) I0507 20:06:15.450752 955 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:15.450779 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:15.632740 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.407s (SENT) I0507 20:06:15.632881 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:15.633325 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.791s (election complete) I0507 20:06:15.633462 955 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:15.639736 955 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:15.639856 955 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.116s (starting election) I0507 20:06:15.639945 955 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:15.640365 955 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:15.640511 955 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:15.641392 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:15.648327 955 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:15.648486 955 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.589s (starting election) I0507 20:06:15.648624 955 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:15.649236 955 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:15.649485 955 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:15.650288 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:15.740133 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.348s (SENT) I0507 20:06:15.740236 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:15.740628 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.828s (election complete) I0507 20:06:15.740821 955 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:15.757319 955 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:15.757436 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.116s (SENT) I0507 20:06:15.757478 955 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.114s (starting election) I0507 20:06:15.757587 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:15.757685 955 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:15.758361 955 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:15.758359 956 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (election complete) I0507 20:06:15.758452 956 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:15.758642 955 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:15.759565 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:15.831696 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:12.830515 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:12.830793 (+ 278us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:12.830811 (+ 18us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:12.830919 (+ 108us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:15.831288 (+3000369us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":82} W0507 20:06:15.832018 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:06:15.873371 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.114s (SENT) I0507 20:06:15.873502 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:15.873883 955 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.542s (election complete) I0507 20:06:15.873965 955 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:16.238683 955 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:16.238813 955 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.788s (starting election) I0507 20:06:16.238940 955 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:16.239388 955 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:16.239591 955 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:16.240860 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.589s (SENT) I0507 20:06:16.240974 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:06:16.241119 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:16.241374 955 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.999s (election complete) I0507 20:06:16.241498 955 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:16.264848 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.813s (SENT) I0507 20:06:16.264962 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:16.265399 955 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.510s (election complete) I0507 20:06:16.265511 955 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:16.569270 955 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:16.569516 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.198s (starting election) I0507 20:06:16.569664 955 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:16.570611 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:16.570847 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:16.571750 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:16.768908 955 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:16.769110 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.813s (starting election) I0507 20:06:16.769242 955 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:16.769857 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:16.769973 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.198s (SENT) I0507 20:06:16.770094 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:16.770133 955 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:16.770303 955 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.330s (election complete) I0507 20:06:16.770368 955 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:16.771049 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:16.776551 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:16.776672 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.342s (starting election) I0507 20:06:16.776775 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:16.777379 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:16.777601 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:16.778648 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:17.029580 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.788s (SENT) I0507 20:06:17.029717 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:17.030095 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.281s (election complete) I0507 20:06:17.030216 959 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:17.101297 959 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:17.101459 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.761s (starting election) I0507 20:06:17.101563 959 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:17.102085 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:17.102282 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:17.103191 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:17.119760 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:17.119891 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.871s (starting election) I0507 20:06:17.120004 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:17.120473 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:17.120697 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:17.120991 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.342s (SENT) I0507 20:06:17.121104 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:17.121320 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.203s (election complete) I0507 20:06:17.121446 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:17.121773 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:17.311630 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:17.311779 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.193s (starting election) I0507 20:06:17.311870 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:17.312403 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:17.312592 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:17.313549 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:17.325783 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:17.325897 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.399s (starting election) I0507 20:06:17.326006 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:17.326459 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:17.326637 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:17.327445 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:17.505755 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:17.505905 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.193s (SENT) I0507 20:06:17.505936 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.694s (starting election) I0507 20:06:17.505996 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:17.506064 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:17.506692 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:17.506703 960 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.939s (election complete) I0507 20:06:17.506798 960 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:17.507019 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:17.508108 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:17.585242 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.813s (SENT) I0507 20:06:17.585409 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:17.585902 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.938s (election complete) I0507 20:06:17.586012 959 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:17.726127 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:17.726197 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.399s (SENT) I0507 20:06:17.726269 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.214s (starting election) I0507 20:06:17.726325 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:17.726402 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:17.727052 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:17.727052 960 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.211s (election complete) I0507 20:06:17.727126 960 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:17.727284 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:17.728237 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:17.864361 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.761s (SENT) I0507 20:06:17.864464 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:17.864889 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.588s (election complete) I0507 20:06:17.865063 959 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:17.939334 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:17.939477 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.359s (starting election) I0507 20:06:17.939566 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:17.940048 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:17.940230 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:17.941182 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:17.941805 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.214s (SENT) I0507 20:06:17.941886 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:17.942148 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.896s (election complete) I0507 20:06:17.942234 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:17.992885 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.871s (SENT) I0507 20:06:17.992985 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:17.993333 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.777s (election complete) I0507 20:06:17.993468 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:18.202996 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.694s (SENT) I0507 20:06:18.203109 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:18.203553 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.400s (election complete) I0507 20:06:18.203698 959 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:18.299806 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.359s (SENT) I0507 20:06:18.299901 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:18.300252 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.234s (election complete) I0507 20:06:18.300374 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:18.454035 959 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:18.454164 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (starting election) I0507 20:06:18.454288 959 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:18.454747 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:18.455003 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:18.456324 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:18.535775 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:18.535938 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.500s (starting election) I0507 20:06:18.536046 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:18.536620 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:18.537739 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:18.538944 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:18.604789 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:18.604914 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.458s (starting election) I0507 20:06:18.605024 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:18.605545 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:18.605741 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:18.606585 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:18.903995 959 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:18.904168 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.787s (starting election) I0507 20:06:18.904285 959 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:18.904799 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:18.905035 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:18.905560 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.449s (SENT) I0507 20:06:18.905730 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:18.906082 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.962s (election complete) I0507 20:06:18.906173 959 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:18.906082 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:18.963609 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:15.962705 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:15.963125 (+ 420us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:15.963146 (+ 21us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:15.963279 (+ 133us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:18.963285 (+3000006us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":98} W0507 20:06:18.963878 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:19.036804 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:19.036931 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.386s (starting election) I0507 20:06:19.037047 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:19.037504 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:19.037716 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:19.037957 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.500s (SENT) I0507 20:06:19.038029 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:19.038197 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.895s (election complete) I0507 20:06:19.038792 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:19.038936 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:19.064316 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.458s (SENT) I0507 20:06:19.064357 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:19.064461 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:19.064522 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.606s (starting election) I0507 20:06:19.064669 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:19.065279 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:19.065311 963 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.489s (election complete) I0507 20:06:19.065397 963 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:19.065608 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:19.066419 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:19.424546 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.386s (SENT) I0507 20:06:19.424650 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:19.425043 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.182s (election complete) I0507 20:06:19.425129 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:19.555923 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:19.556071 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.167s (starting election) I0507 20:06:19.556198 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:19.556764 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:19.557000 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:19.557940 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:19.608064 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:19.608239 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.166s (starting election) I0507 20:06:19.608345 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:19.608885 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:19.609089 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:19.610021 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:19.672516 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.606s (SENT) I0507 20:06:19.672693 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:19.673105 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.592s (election complete) I0507 20:06:19.673208 959 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:19.693676 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.787s (SENT) I0507 20:06:19.693814 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:19.694291 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.387s (election complete) I0507 20:06:19.694425 959 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:19.725208 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.167s (SENT) I0507 20:06:19.725327 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:19.725710 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.159s (election complete) I0507 20:06:19.725852 959 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:19.775565 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:19.775624 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.166s (SENT) I0507 20:06:19.775769 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.559s (starting election) I0507 20:06:19.775851 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:19.775925 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:19.777456 965 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.164s (election complete) I0507 20:06:19.777477 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:19.777576 965 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:19.777825 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:19.778760 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:19.885283 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:19.885458 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.962s (starting election) I0507 20:06:19.885583 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:19.886201 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:19.886451 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:19.887591 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:19.942420 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:19.942564 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.536s (starting election) I0507 20:06:19.942685 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:19.943250 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:19.943466 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:19.944492 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:20.082275 959 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:20.082442 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.297s (starting election) I0507 20:06:20.082550 959 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:20.083155 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:20.083360 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:20.084417 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:20.337724 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.559s (SENT) I0507 20:06:20.337832 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:20.338153 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.314s (election complete) I0507 20:06:20.338291 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:20.381002 959 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:20.381150 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.133s (starting election) W0507 20:06:20.382407 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.297s (SENT) I0507 20:06:20.382542 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:20.382395 959 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:20.383096 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:20.383216 966 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.633s (election complete) I0507 20:06:20.383289 966 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:20.383352 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:20.384310 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:20.480188 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.536s (SENT) I0507 20:06:20.480303 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:20.480636 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.172s (election complete) I0507 20:06:20.480784 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:20.517184 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.133s (SENT) I0507 20:06:20.517298 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:20.517719 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.379s (election complete) I0507 20:06:20.517865 959 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:20.653591 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:20.653774 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.770s (starting election) I0507 20:06:20.653885 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:20.654443 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:20.654655 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:20.655513 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:20.848219 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:20.848450 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.268s (starting election) I0507 20:06:20.848587 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:20.849400 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:20.849629 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:20.850100 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.962s (SENT) I0507 20:06:20.850232 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:20.850450 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.310s (election complete) W0507 20:06:20.850553 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:20.850726 959 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:20.898197 959 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:20.898380 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.199s (starting election) I0507 20:06:20.898506 959 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:20.899080 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:20.899264 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:20.900089 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:21.098863 959 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:21.099023 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.351s (starting election) W0507 20:06:21.099196 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.199s (SENT) I0507 20:06:21.099290 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:21.099154 959 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:21.099611 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:21.099885 959 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:21.099992 967 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.262s (election complete) I0507 20:06:21.100078 967 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:21.100770 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:21.119045 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.268s (SENT) I0507 20:06:21.119179 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:21.119592 967 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.528s (election complete) I0507 20:06:21.119700 967 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:21.363289 967 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:21.363435 967 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.107s (starting election) I0507 20:06:21.363536 967 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:21.363994 967 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:21.364223 967 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:21.365386 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:21.425149 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.770s (SENT) I0507 20:06:21.425161 967 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:21.425259 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:21.425307 967 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.568s (starting election) I0507 20:06:21.425410 967 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:21.425902 967 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:21.425976 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.540s (election complete) I0507 20:06:21.426086 967 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:21.426204 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:21.427207 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:21.451666 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.351s (SENT) I0507 20:06:21.451819 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:21.452266 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.825s (election complete) I0507 20:06:21.452397 959 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:21.471726 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.107s (SENT) I0507 20:06:21.471880 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:21.472369 959 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.939s (election complete) I0507 20:06:21.472498 959 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:21.648453 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:21.648591 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.646s (starting election) I0507 20:06:21.648684 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:21.649148 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:21.649346 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:21.650384 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:21.967778 959 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:21.967957 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.211s (starting election) I0507 20:06:21.968163 959 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:21.968873 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:21.969240 959 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:21.970360 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:21.994611 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.568s (SENT) I0507 20:06:21.994742 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:21.995231 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.211s (election complete) I0507 20:06:21.995448 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:22.099586 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:19.101949 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:19.102259 (+ 310us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:19.102276 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:19.102450 (+ 174us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:22.099181 (+2996731us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":88} W0507 20:06:22.099911 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:06:22.180969 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.211s (SENT) I0507 20:06:22.181076 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:22.181475 959 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.544s (election complete) I0507 20:06:22.181602 959 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:22.295717 959 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:22.295909 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.209s (starting election) I0507 20:06:22.296046 959 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:22.296758 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.646s (SENT) I0507 20:06:22.296797 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:22.296862 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:22.297096 959 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:22.297317 959 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.609s (election complete) I0507 20:06:22.297385 959 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:22.298213 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:22.412680 970 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:22.412837 970 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.632s (starting election) I0507 20:06:22.412935 970 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:22.413457 970 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:22.413684 970 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:22.414862 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:22.507436 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.209s (SENT) I0507 20:06:22.507560 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:22.507975 970 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.818s (election complete) I0507 20:06:22.508129 970 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:22.726327 970 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:22.726536 970 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.960s (starting election) I0507 20:06:22.726701 970 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:22.727358 970 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:22.727622 970 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:22.728663 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:23.046286 970 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:23.046437 970 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.685s (starting election) W0507 20:06:23.046377 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.632s (SENT) I0507 20:06:23.046689 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:23.046612 970 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:23.047158 970 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:23.047423 970 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:23.047662 970 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.323s (election complete) I0507 20:06:23.047770 970 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:23.048313 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:23.327379 971 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:23.327508 971 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.565s (starting election) I0507 20:06:23.327610 971 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:23.328086 971 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:23.328321 971 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:23.329255 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:23.371767 971 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:23.371914 971 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.556s (starting election) I0507 20:06:23.372014 971 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:23.372510 971 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:23.372681 971 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:23.373525 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:23.687209 971 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:23.687357 971 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.918s (starting election) I0507 20:06:23.687458 971 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:23.687916 971 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:23.688117 971 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:23.689095 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:23.689628 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.960s (SENT) I0507 20:06:23.689697 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:23.689932 971 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.400s (election complete) I0507 20:06:23.690027 971 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:23.734273 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.685s (SENT) I0507 20:06:23.734393 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:23.734746 971 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.463s (election complete) I0507 20:06:23.734874 971 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:23.893569 971 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:23.893745 971 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.296s (starting election) I0507 20:06:23.893853 971 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:23.894402 971 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:23.894635 971 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:23.894692 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.565s (SENT) I0507 20:06:23.894812 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:23.895088 971 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.782s (election complete) I0507 20:06:23.895165 971 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:23.895608 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:23.930014 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.556s (SENT) I0507 20:06:23.930130 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:23.930488 971 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.890s (election complete) I0507 20:06:23.930574 971 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:24.090873 971 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:24.091030 971 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.376s (starting election) I0507 20:06:24.091156 971 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:24.091683 971 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:24.091907 971 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:24.093049 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:24.192071 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.296s (SENT) I0507 20:06:24.192198 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:24.192548 971 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.693s (election complete) I0507 20:06:24.192667 971 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:24.468530 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.376s (SENT) I0507 20:06:24.468530 971 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:24.468647 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:24.468693 971 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.976s (starting election) I0507 20:06:24.468786 971 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:24.470072 971 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:24.470310 971 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:24.470489 971 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.861s (election complete) I0507 20:06:24.470589 971 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:24.471374 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:24.608072 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.918s (SENT) I0507 20:06:24.608198 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:24.608654 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.681s (election complete) I0507 20:06:24.608777 972 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:24.821408 972 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:24.821576 972 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.598s (starting election) I0507 20:06:24.821723 972 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:24.822289 972 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:24.822489 972 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:24.823258 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:24.886420 972 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:24.886606 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (starting election) I0507 20:06:24.886708 972 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:24.887235 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:24.887454 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:24.888314 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:25.247485 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:22.247241 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:22.247575 (+ 334us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:22.247589 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:22.247687 (+ 98us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:25.247183 (+2999496us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":101} W0507 20:06:25.247725 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:25.291149 972 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:25.291271 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.359s (starting election) I0507 20:06:25.291378 972 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:25.291854 972 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:25.292047 972 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:25.292907 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:25.369822 972 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:25.370010 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.827s (starting election) I0507 20:06:25.370113 972 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:25.370659 972 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:25.370853 972 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:25.371917 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:25.420727 972 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:25.420884 972 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.540s (starting election) I0507 20:06:25.421070 972 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:25.421670 972 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:25.421896 972 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:25.422974 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.598s (SENT) W0507 20:06:25.423032 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:25.423118 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:25.423424 972 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.997s (election complete) I0507 20:06:25.423506 972 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:25.448640 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.976s (SENT) I0507 20:06:25.448750 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:25.449175 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.372s (election complete) I0507 20:06:25.449326 972 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:25.651983 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.359s (SENT) I0507 20:06:25.652093 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:25.652473 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.527s (election complete) I0507 20:06:25.652634 972 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:25.750713 972 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:25.750895 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.441s (starting election) I0507 20:06:25.751008 972 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:25.751559 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:25.751724 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.863s (SENT) I0507 20:06:25.751818 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:25.751839 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:25.752127 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.262s (election complete) I0507 20:06:25.752213 972 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:25.752699 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:25.964617 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.540s (SENT) I0507 20:06:25.964754 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:25.965169 972 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.426s (election complete) I0507 20:06:25.965271 972 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:26.014946 972 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:26.015084 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.475s (starting election) I0507 20:06:26.015194 972 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:26.015738 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:26.015957 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:26.017134 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:26.180965 972 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:26.181131 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.187s (starting election) I0507 20:06:26.181236 972 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:26.181727 972 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:26.181946 972 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:26.182996 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:26.194255 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.441s (SENT) I0507 20:06:26.194408 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:26.194803 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.230s (election complete) I0507 20:06:26.194948 972 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:26.198921 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.827s (SENT) I0507 20:06:26.199024 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:26.199353 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.606s (election complete) I0507 20:06:26.199465 972 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:26.369647 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.187s (SENT) I0507 20:06:26.369766 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:26.370220 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.512s (election complete) I0507 20:06:26.370327 972 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:26.391649 972 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:26.391789 972 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.601s (starting election) I0507 20:06:26.391896 972 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:26.392427 972 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:26.392614 972 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:26.393385 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:26.426241 972 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:26.426431 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.754s (starting election) I0507 20:06:26.426540 972 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:26.427284 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:26.427562 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:26.428714 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:26.491803 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.475s (SENT) I0507 20:06:26.491955 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:26.492348 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.238s (election complete) I0507 20:06:26.492446 972 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:26.731348 972 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:26.731513 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.440s (starting election) I0507 20:06:26.731667 972 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:26.732383 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:26.732666 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:26.739792 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:26.883196 972 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:26.883385 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.481s (starting election) I0507 20:06:26.883491 972 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:26.884012 972 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:26.884246 972 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:26.885401 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:26.993549 972 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:26.993672 972 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.189s (starting election) I0507 20:06:26.993784 972 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:26.994328 972 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:26.994520 972 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:26.994951 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.601s (SENT) I0507 20:06:26.995045 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:26.995376 972 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.799s (election complete) I0507 20:06:26.995537 972 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:26.995542 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:27.172987 972 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:27.173159 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.802s (starting election) I0507 20:06:27.173307 972 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:27.173734 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.440s (SENT) I0507 20:06:27.173837 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:27.173897 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:27.174123 972 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:27.174280 972 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.756s (election complete) I0507 20:06:27.174355 972 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:27.175197 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:27.182816 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.754s (SENT) I0507 20:06:27.182919 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:27.183275 975 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.923s (election complete) I0507 20:06:27.183393 975 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:27.184425 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.189s (SENT) I0507 20:06:27.184494 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:27.184689 975 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.135s (election complete) I0507 20:06:27.184777 975 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:27.320772 975 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:27.320943 975 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.622s (starting election) I0507 20:06:27.321082 975 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:27.321671 975 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:27.321933 975 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:27.322816 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:27.365694 975 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:27.365777 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.481s (SENT) I0507 20:06:27.365880 975 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.567s (starting election) I0507 20:06:27.365918 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:27.366042 975 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:27.366701 975 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:27.366709 972 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.497s (election complete) I0507 20:06:27.366799 972 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:27.366928 975 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:27.367895 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:27.864825 975 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:27.864990 975 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.991s (starting election) I0507 20:06:27.865136 975 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:27.865768 975 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:27.866051 975 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:27.867290 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:27.935292 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.567s (SENT) I0507 20:06:27.935405 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:27.935808 975 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.132s (election complete) I0507 20:06:27.935973 975 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:27.943552 975 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:27.943753 975 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.509s (starting election) I0507 20:06:27.943864 975 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:27.944473 975 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:27.944700 975 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:27.945521 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:27.946322 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.622s (SENT) I0507 20:06:27.946391 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:27.946633 975 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.796s (election complete) I0507 20:06:27.946717 975 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:27.977206 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.802s (SENT) I0507 20:06:27.977339 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:27.977766 975 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.429s (election complete) I0507 20:06:27.977891 975 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:28.068572 975 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:28.068759 975 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.981s (starting election) I0507 20:06:28.068917 975 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:28.069535 975 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:28.069861 975 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:28.070998 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:28.395639 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:25.397975 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:25.398255 (+ 280us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:25.398270 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:25.398368 (+ 98us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:28.395228 (+2996860us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":104} W0507 20:06:28.395957 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:28.407305 975 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:28.407457 975 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.395s (starting election) I0507 20:06:28.407562 975 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:28.408090 975 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:28.408299 975 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:28.409240 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:28.455170 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.509s (SENT) I0507 20:06:28.455265 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:28.455624 975 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.192s (election complete) I0507 20:06:28.455739 975 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:28.648785 975 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:28.648926 975 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.453s (starting election) I0507 20:06:28.649039 975 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:28.649538 975 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:28.649758 975 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:28.651132 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:28.803452 975 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:28.803638 975 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (starting election) W0507 20:06:28.803783 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.395s (SENT) I0507 20:06:28.803885 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:28.803768 975 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:28.804467 975 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:28.804519 978 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.138s (election complete) I0507 20:06:28.804610 978 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:28.804756 975 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:28.805616 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:28.858103 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.991s (SENT) I0507 20:06:28.858220 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:28.858660 975 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.404s (election complete) I0507 20:06:28.858777 975 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:28.944103 975 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:28.944258 975 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.704s (starting election) I0507 20:06:28.944372 975 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:28.944864 975 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:28.945104 975 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:28.946084 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:29.052315 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.981s (SENT) I0507 20:06:29.052435 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:29.052866 975 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.590s (election complete) I0507 20:06:29.053011 975 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:29.102399 975 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:29.102555 975 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.454s (starting election) I0507 20:06:29.102663 975 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:29.103312 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.453s (SENT) I0507 20:06:29.103366 975 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:29.103417 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:29.103585 975 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:29.103797 978 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.723s (election complete) I0507 20:06:29.103925 978 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:29.104326 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:29.558701 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.454s (SENT) I0507 20:06:29.558833 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:29.559273 978 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.766s (election complete) I0507 20:06:29.559389 978 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:29.643502 978 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:29.643662 978 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.421s (starting election) I0507 20:06:29.643817 978 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:29.644376 978 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:29.644641 978 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:29.645995 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:29.648965 978 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:29.649065 978 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (starting election) I0507 20:06:29.649159 978 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:29.649624 978 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:29.649806 978 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:29.651237 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.704s (SENT) I0507 20:06:29.651365 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:06:29.651445 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:29.651726 978 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.204s (election complete) I0507 20:06:29.651840 978 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:29.669428 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.863s (SENT) I0507 20:06:29.669534 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:29.669997 978 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.870s (election complete) I0507 20:06:29.670125 978 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:30.065970 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.421s (SENT) I0507 20:06:30.065963 978 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:30.066123 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:30.066161 978 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.905s (starting election) I0507 20:06:30.066288 978 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:30.066800 980 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.489s (election complete) I0507 20:06:30.066824 978 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:30.066961 980 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:30.067227 978 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:30.068132 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:30.099522 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.449s (SENT) I0507 20:06:30.099642 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:30.099999 978 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.144s (election complete) I0507 20:06:30.100107 978 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:30.245491 978 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:30.245648 978 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.401s (starting election) I0507 20:06:30.245748 978 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:30.246242 978 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:30.246428 978 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:30.247539 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:30.325966 978 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:30.326107 978 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.667s (starting election) I0507 20:06:30.326208 978 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:30.326746 978 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:30.326941 978 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:30.328089 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:30.557025 978 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:30.557185 978 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.568s (starting election) I0507 20:06:30.557282 978 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:30.557790 978 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:30.558009 978 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:30.559180 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:30.648174 978 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:30.648304 978 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.705s (starting election) W0507 20:06:30.648521 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.401s (SENT) I0507 20:06:30.648432 978 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:30.648633 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:30.648972 978 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:30.649286 978 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:30.649272 981 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.237s (election complete) I0507 20:06:30.649356 981 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:30.650076 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:30.887245 981 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:30.887357 981 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.395s (starting election) I0507 20:06:30.887487 981 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:30.887989 981 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:30.888200 981 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:30.889261 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:30.974148 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.905s (SENT) I0507 20:06:30.974284 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:30.974633 981 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.498s (election complete) I0507 20:06:30.974783 981 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:30.994194 981 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:30.994371 981 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.533s (starting election) I0507 20:06:30.994478 981 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:30.995077 981 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:30.995275 981 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:30.995573 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.667s (SENT) I0507 20:06:30.995636 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:30.996008 981 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.808s (election complete) I0507 20:06:30.996150 981 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:30.996402 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:31.126993 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.568s (SENT) I0507 20:06:31.127108 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:31.127446 981 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.101s (election complete) I0507 20:06:31.127527 981 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:31.228811 981 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:31.228921 981 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (starting election) I0507 20:06:31.229063 981 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:31.229527 981 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:31.229717 981 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:31.230823 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:31.283648 981 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:31.283805 981 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.528s (starting election) I0507 20:06:31.283900 981 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:31.284296 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.395s (SENT) I0507 20:06:31.284373 981 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:31.284379 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:31.284536 981 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:31.284662 981 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.401s (election complete) I0507 20:06:31.284720 981 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:31.285457 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:31.355126 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.705s (SENT) I0507 20:06:31.355228 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:31.355664 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.208s (election complete) I0507 20:06:31.355811 982 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:31.529708 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.533s (SENT) I0507 20:06:31.529820 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:31.530270 982 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.510s (election complete) I0507 20:06:31.530393 982 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:31.551044 982 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:31.551089 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.321s (SENT) I0507 20:06:31.551206 982 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (starting election) I0507 20:06:31.551242 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:31.551354 982 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:31.551980 982 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:31.552045 981 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.372s (election complete) I0507 20:06:31.552134 981 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:31.552345 982 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:31.553233 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:31.555387 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:28.557270 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:28.557606 (+ 336us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:28.557627 (+ 21us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:28.557773 (+ 146us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:31.555138 (+2997365us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":94} W0507 20:06:31.555618 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:31.565124 982 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:31.565284 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (starting election) I0507 20:06:31.565383 982 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:31.565861 982 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:31.566072 982 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:31.566846 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:31.665848 982 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:31.666049 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.492s (starting election) W0507 20:06:31.666375 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.099s (SENT) I0507 20:06:31.666213 982 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:31.666468 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:31.666941 982 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:31.667017 981 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.536s (election complete) I0507 20:06:31.667094 981 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:31.667177 982 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:31.668120 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:31.814939 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.528s (SENT) I0507 20:06:31.815043 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:31.815454 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.132s (election complete) I0507 20:06:31.815599 982 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:31.874352 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.321s (SENT) I0507 20:06:31.874452 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:31.874900 982 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.627s (election complete) I0507 20:06:31.875064 982 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:31.948539 982 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:31.948734 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.803s (starting election) I0507 20:06:31.948858 982 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:31.949378 982 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:31.949591 982 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:31.950320 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:32.041630 982 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:32.041817 982 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.666s (starting election) I0507 20:06:32.041965 982 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:32.042686 982 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:32.043011 982 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:32.043818 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:32.162009 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.492s (SENT) I0507 20:06:32.162113 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:32.162621 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.658s (election complete) I0507 20:06:32.162813 982 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:32.502873 982 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:32.503074 982 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.795s (starting election) I0507 20:06:32.503186 982 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:32.503693 982 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:32.503933 982 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:32.504878 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:32.708500 982 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:32.708640 982 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.335s (starting election) I0507 20:06:32.708756 982 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:32.709290 982 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:32.709506 982 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:32.709834 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.666s (SENT) I0507 20:06:32.709916 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:32.710135 982 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.927s (election complete) I0507 20:06:32.710242 982 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:32.710383 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:32.754125 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.803s (SENT) I0507 20:06:32.754231 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:32.754587 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.256s (election complete) I0507 20:06:32.754689 982 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:33.011821 982 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:33.011975 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.778s (starting election) I0507 20:06:33.012091 982 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:33.012565 982 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:33.012737 982 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:33.013540 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:33.045472 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.335s (SENT) I0507 20:06:33.045601 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:33.046077 982 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.160s (election complete) I0507 20:06:33.046190 982 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:33.207267 982 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:33.207376 982 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.828s (starting election) I0507 20:06:33.207461 982 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:33.207888 982 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:33.208050 982 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:33.208791 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:33.299311 982 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:06:33.299387 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.795s (SENT) I0507 20:06:33.299437 982 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.207s (starting election) I0507 20:06:33.299468 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:33.299515 982 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:33.300037 982 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:33.300206 982 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:33.300348 985 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.733s (election complete) I0507 20:06:33.300480 985 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:33.301108 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:33.507807 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.207s (SENT) I0507 20:06:33.507930 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:33.508430 985 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.324s (election complete) I0507 20:06:33.508551 985 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:33.790763 985 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:33.790954 985 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.480s (starting election) I0507 20:06:33.791085 985 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:33.791663 985 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:33.791820 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.778s (SENT) I0507 20:06:33.791913 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:33.792201 982 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.514s (election complete) I0507 20:06:33.792285 982 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:33.793428 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:33.793437 985 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:33.833711 985 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:33.833920 985 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.299s (starting election) I0507 20:06:33.834028 985 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:33.834547 985 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:33.834769 985 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:33.835721 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:34.036434 985 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:34.036589 985 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.524s (starting election) I0507 20:06:34.036725 985 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:34.037222 985 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:34.037422 985 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:34.037786 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.828s (SENT) I0507 20:06:34.037947 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:34.038233 985 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.903s (election complete) W0507 20:06:34.038245 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:34.038391 985 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:34.133862 985 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:34.134083 985 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.247s (starting election) I0507 20:06:34.134225 985 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } W0507 20:06:34.134944 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.299s (SENT) I0507 20:06:34.135017 985 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:34.135046 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:34.135324 985 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:34.135391 982 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.659s (election complete) I0507 20:06:34.135493 982 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:34.136507 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:34.274734 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.480s (SENT) I0507 20:06:34.274840 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:34.275424 985 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.861s (election complete) I0507 20:06:34.275584 985 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:34.382663 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.247s (SENT) I0507 20:06:34.382766 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:34.383218 985 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.300s (election complete) I0507 20:06:34.383374 985 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:34.562918 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.524s (SENT) I0507 20:06:34.563050 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:34.563621 985 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.515s (election complete) I0507 20:06:34.563762 985 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:34.684178 985 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:34.684340 985 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.994s (starting election) I0507 20:06:34.684480 985 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:34.685001 985 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:34.685245 985 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:34.686939 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:34.731513 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:31.730539 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:31.730825 (+ 286us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:31.730842 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:31.730958 (+ 116us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:34.731175 (+3000217us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":68} W0507 20:06:34.731781 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:35.080091 985 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:35.080271 985 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.760s (starting election) I0507 20:06:35.080391 985 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:35.080930 985 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:35.081115 985 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:35.082024 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:35.137670 985 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:35.137797 985 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.156s (starting election) I0507 20:06:35.137902 985 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:35.138345 985 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:35.138545 985 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:35.139591 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:35.294842 985 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:35.294998 985 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.918s (starting election) W0507 20:06:35.295063 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.156s (SENT) I0507 20:06:35.295147 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:35.295078 985 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:35.295579 985 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:35.295840 985 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:35.296159 989 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.276s (election complete) I0507 20:06:35.296241 989 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:35.296741 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:35.573686 989 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:35.573812 989 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.997s (starting election) I0507 20:06:35.573920 989 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:35.574422 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:35.574633 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:35.575606 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:35.679322 989 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:35.679533 989 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.357s (starting election) I0507 20:06:35.679656 989 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:35.680207 989 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:35.680423 989 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:35.680804 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.994s (SENT) I0507 20:06:35.680980 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:35.681299 985 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.851s (election complete) I0507 20:06:35.681378 985 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:35.681624 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:35.841140 989 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:35.841269 989 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.723s (starting election) I0507 20:06:35.841375 989 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:35.841857 989 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:35.842068 989 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:35.842589 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.760s (SENT) I0507 20:06:35.843796 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:06:35.843791 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:35.844197 989 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.670s (election complete) I0507 20:06:35.844307 989 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:36.037773 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.357s (SENT) I0507 20:06:36.037904 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:36.038311 989 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.450s (election complete) I0507 20:06:36.038475 989 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:36.215276 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.918s (SENT) I0507 20:06:36.215412 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:36.215790 989 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.138s (election complete) I0507 20:06:36.215924 989 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:36.354408 989 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:36.354537 989 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.148s (starting election) I0507 20:06:36.354630 989 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:36.355129 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:36.355310 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:36.356554 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:36.489147 989 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:36.489326 989 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.204s (starting election) I0507 20:06:36.489524 989 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:36.490125 989 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:36.490372 989 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:36.491405 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:36.503870 989 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:36.504032 989 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.110s (starting election) W0507 20:06:36.505201 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.148s (SENT) I0507 20:06:36.505105 989 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:36.505316 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:36.505861 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:36.506182 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:36.506233 991 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.864s (election complete) I0507 20:06:36.506471 991 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:36.507158 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:36.515041 991 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:36.515175 991 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.594s (starting election) I0507 20:06:36.515314 991 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:36.516003 991 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:36.516266 991 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:36.517050 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:36.565968 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.723s (SENT) I0507 20:06:36.566085 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:36.566426 991 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.826s (election complete) I0507 20:06:36.566560 991 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:36.573896 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.997s (SENT) I0507 20:06:36.574004 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:36.574276 991 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.113s (election complete) I0507 20:06:36.574373 991 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:36.617092 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.110s (SENT) I0507 20:06:36.617224 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:36.617599 991 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.750s (election complete) I0507 20:06:36.617723 991 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:36.694689 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.204s (SENT) I0507 20:06:36.694784 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:36.695096 989 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:36.695184 989 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.580s (starting election) I0507 20:06:36.695307 989 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:36.695736 989 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:36.695777 991 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.959s (election complete) I0507 20:06:36.695869 991 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:36.695922 989 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:36.696707 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:37.112854 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.594s (SENT) I0507 20:06:37.112994 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:37.113353 989 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.635s (election complete) I0507 20:06:37.113456 989 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:37.276672 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.580s (SENT) I0507 20:06:37.276849 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:37.277245 989 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.812s (election complete) I0507 20:06:37.277382 989 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:37.369225 989 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:37.369415 989 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.232s (starting election) I0507 20:06:37.369536 989 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:37.370128 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:37.370352 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:37.371793 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:37.603029 989 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:37.603207 989 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.397s (starting election) W0507 20:06:37.603317 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.232s (SENT) I0507 20:06:37.603471 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:37.603340 989 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:37.603938 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:37.604226 989 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:37.604409 993 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.819s (election complete) I0507 20:06:37.604588 993 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:37.605418 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:37.749053 993 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:37.749223 993 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.162s (starting election) I0507 20:06:37.749362 993 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:37.749979 993 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:37.750264 993 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:37.751353 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:37.911500 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:34.911574 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:34.911812 (+ 238us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:34.911826 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:34.911917 (+ 91us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:37.911229 (+2999312us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":69} W0507 20:06:37.911931 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:37.912308 993 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:37.912431 993 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.354s (starting election) I0507 20:06:37.912528 993 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:37.912994 993 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:37.913170 993 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:37.913478 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.162s (SENT) I0507 20:06:37.913537 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:37.913741 993 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.595s (election complete) I0507 20:06:37.913822 993 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:37.914309 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:38.001771 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.397s (SENT) I0507 20:06:38.001879 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:38.002332 993 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.849s (election complete) I0507 20:06:38.002461 993 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:38.089968 993 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:38.090147 993 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.825s (starting election) I0507 20:06:38.090306 993 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:38.091048 993 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:38.091302 993 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:38.092471 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:38.267655 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.354s (SENT) I0507 20:06:38.267774 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:38.268213 993 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.420s (election complete) I0507 20:06:38.268409 993 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:38.689687 993 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:38.689967 993 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.086s (starting election) I0507 20:06:38.690127 993 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:38.690925 993 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:38.691223 993 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:38.692549 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:38.777624 993 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:38.777802 993 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.322s (starting election) I0507 20:06:38.778000 993 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:38.779089 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.086s (SENT) I0507 20:06:38.779181 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:38.779283 993 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:38.779760 993 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:38.780154 997 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (election complete) W0507 20:06:38.780534 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:38.780613 997 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:38.852176 997 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:38.852455 997 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.745s (starting election) I0507 20:06:38.852627 997 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:38.853233 997 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:38.853482 997 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:38.854476 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:38.880475 997 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:38.880635 997 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.904s (starting election) I0507 20:06:38.880741 997 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:38.881327 997 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:38.881582 997 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:38.882681 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:38.916219 997 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:38.916354 997 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.453s (starting election) I0507 20:06:38.916465 997 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:38.917021 997 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:38.917273 997 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:38.917712 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.825s (SENT) I0507 20:06:38.917842 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:38.918195 997 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.192s (election complete) I0507 20:06:38.918282 997 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:38.918396 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:39.102923 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.322s (SENT) I0507 20:06:39.103044 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:39.103400 997 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.463s (election complete) I0507 20:06:39.103536 997 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:39.111539 997 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:39.111675 997 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.820s (starting election) I0507 20:06:39.111770 997 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:39.112241 997 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:39.112421 997 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:39.113659 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:39.371533 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.453s (SENT) I0507 20:06:39.371656 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:39.372112 997 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.674s (election complete) I0507 20:06:39.372233 997 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:39.568078 997 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:39.568224 997 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.942s (starting election) I0507 20:06:39.568336 997 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:39.568981 997 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:39.569279 997 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:39.570346 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:39.598736 997 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:39.598940 997 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.259s (starting election) I0507 20:06:39.599107 997 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:39.599809 997 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:39.599812 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.745s (SENT) I0507 20:06:39.599951 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:39.600103 997 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:39.600313 997 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.691s (election complete) I0507 20:06:39.600383 997 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:39.601184 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:39.787533 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.904s (SENT) I0507 20:06:39.787665 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:39.788139 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.515s (election complete) I0507 20:06:39.788281 998 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:39.860630 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.259s (SENT) I0507 20:06:39.860764 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:39.861186 998 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.817s (election complete) I0507 20:06:39.861277 998 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:39.934001 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.820s (SENT) I0507 20:06:39.934146 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:39.934648 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.866s (election complete) I0507 20:06:39.934762 998 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:40.304234 998 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:40.304383 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (starting election) I0507 20:06:40.304520 998 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:40.305167 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:40.305423 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:40.307065 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:40.512578 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.942s (SENT) I0507 20:06:40.512686 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:40.513062 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.395s (election complete) I0507 20:06:40.513324 998 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:40.679278 998 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:40.679426 998 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.591s (starting election) I0507 20:06:40.679517 998 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:40.679975 998 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:40.680167 998 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:40.680891 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:40.801857 998 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:40.801993 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.097s (starting election) I0507 20:06:40.802137 998 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:40.802618 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:40.802841 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:40.803895 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:40.899510 998 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:40.899662 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.686s (starting election) I0507 20:06:40.899767 998 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:40.900247 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:40.900444 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:40.900820 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.097s (SENT) I0507 20:06:40.900923 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:40.901279 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.453s (election complete) I0507 20:06:40.901408 998 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:40.901487 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:40.908578 998 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:40.908715 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.107s (starting election) I0507 20:06:40.908823 998 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:40.909346 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:40.909538 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:40.910379 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:41.016475 998 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:41.016620 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.717s (starting election) I0507 20:06:41.016712 998 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:41.017033 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.107s (SENT) I0507 20:06:41.017132 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:41.017362 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:41.018077 1000 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.342s (election complete) I0507 20:06:41.018152 1000 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:41.018347 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:41.018661 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:41.023304 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.716s (SENT) I0507 20:06:41.023399 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:41.023687 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.685s (election complete) I0507 20:06:41.023766 998 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:41.103667 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:38.102563 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:38.102854 (+ 291us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:38.102870 (+ 16us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:38.102981 (+ 111us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:41.103293 (+3000312us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":87} W0507 20:06:41.103952 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:41.271025 998 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:41.271328 998 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.601s (starting election) I0507 20:06:41.271450 998 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:41.271922 998 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:41.272104 998 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:41.273092 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:41.273159 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.591s (SENT) I0507 20:06:41.273226 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:41.273470 998 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.219s (election complete) I0507 20:06:41.273573 998 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:41.355532 998 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:41.355682 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.307s (starting election) I0507 20:06:41.355772 998 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:41.356365 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:41.356544 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:41.357556 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:41.494081 998 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:41.494244 998 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.700s (starting election) I0507 20:06:41.494361 998 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:41.495024 998 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:41.495352 998 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:41.496259 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:41.588590 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.686s (SENT) I0507 20:06:41.588706 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:41.589100 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.718s (election complete) I0507 20:06:41.589263 998 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:41.663964 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.307s (SENT) I0507 20:06:41.664089 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:41.664499 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.187s (election complete) I0507 20:06:41.664655 998 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:41.709837 998 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:41.710017 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.772s (starting election) I0507 20:06:41.710144 998 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:41.710793 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:41.711086 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:41.711942 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:41.736882 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.717s (SENT) I0507 20:06:41.737011 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:41.737423 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.510s (election complete) I0507 20:06:41.737505 998 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:41.852170 998 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:41.852360 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.594s (starting election) I0507 20:06:41.852470 998 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:41.852957 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:41.853165 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:41.854195 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:41.874742 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.601s (SENT) I0507 20:06:41.874862 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:41.875252 998 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.710s (election complete) I0507 20:06:41.875365 998 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:42.197005 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.700s (SENT) I0507 20:06:42.197151 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:42.197602 998 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.701s (election complete) I0507 20:06:42.197813 998 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:42.248260 998 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:42.248477 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.235s (starting election) I0507 20:06:42.248646 998 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:42.249471 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:42.249764 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:42.250942 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:42.447015 998 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:42.447154 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.153s (starting election) I0507 20:06:42.447269 998 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:42.447784 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:42.447999 998 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:42.448148 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.594s (SENT) I0507 20:06:42.448259 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:42.448544 998 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.652s (election complete) I0507 20:06:42.448637 998 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:42.448973 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:42.484678 998 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:42.484974 998 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.755s (starting election) W0507 20:06:42.484979 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.772s (SENT) I0507 20:06:42.485065 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:42.485082 998 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:42.485682 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:42.485886 998 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:42.485846 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.439s (election complete) I0507 20:06:42.485970 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:42.486217 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.235s (SENT) I0507 20:06:42.486281 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:42.486495 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.689s (election complete) I0507 20:06:42.486594 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:42.486826 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:42.602324 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.153s (SENT) I0507 20:06:42.602455 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:42.602823 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.533s (election complete) I0507 20:06:42.602933 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:42.899974 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:42.900115 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.706s (starting election) I0507 20:06:42.900233 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:42.900733 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:42.900905 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:42.901799 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:43.137009 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:43.137212 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.143s (starting election) I0507 20:06:43.137315 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:43.137799 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:43.137995 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:43.139087 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:43.176187 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:43.176322 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.430s (starting election) I0507 20:06:43.176419 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:43.176892 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:43.177090 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:43.177999 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:43.242504 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.755s (SENT) I0507 20:06:43.242669 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:43.243185 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.803s (election complete) I0507 20:06:43.243312 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:43.280819 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:43.281018 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.137s (starting election) W0507 20:06:43.282166 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.143s (SENT) I0507 20:06:43.282279 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:43.282155 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:43.283027 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:43.283195 1004 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.548s (election complete) I0507 20:06:43.283357 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:43.283363 1004 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:43.284499 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:43.421459 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.137s (SENT) I0507 20:06:43.421643 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:43.422097 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.975s (election complete) I0507 20:06:43.422210 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:43.606676 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:43.606828 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.579s (starting election) I0507 20:06:43.607004 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:43.607569 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:43.607812 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:43.607852 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.430s (SENT) I0507 20:06:43.607925 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:43.608141 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.806s (election complete) I0507 20:06:43.608234 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:43.608317 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.706s (SENT) I0507 20:06:43.608364 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:43.608484 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.125s (election complete) I0507 20:06:43.608541 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:43.608770 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:43.734957 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:43.735113 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.673s (starting election) I0507 20:06:43.735209 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:43.735740 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:43.735934 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:43.736793 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:44.188371 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.579s (SENT) I0507 20:06:44.188508 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:44.188946 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.131s (election complete) I0507 20:06:44.189055 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:44.303611 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:41.304615 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:41.304995 (+ 380us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:41.305012 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:41.305145 (+ 133us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:44.303320 (+2998175us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":108} W0507 20:06:44.303900 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:44.321424 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:44.321564 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.774s (starting election) I0507 20:06:44.321696 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:44.322240 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:44.322449 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:44.324105 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:44.397635 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:44.397785 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.618s (starting election) I0507 20:06:44.397883 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:44.398358 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:44.398538 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:44.399649 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:44.410514 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.673s (SENT) I0507 20:06:44.410645 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:44.411170 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.127s (election complete) I0507 20:06:44.411296 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:44.415643 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:44.415798 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.307s (starting election) I0507 20:06:44.415964 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:44.416734 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:44.417047 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:44.418057 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:44.538944 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:44.539126 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (starting election) I0507 20:06:44.539232 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:44.539788 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:44.540004 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:44.541069 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:44.723647 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:44.723801 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.178s (starting election) I0507 20:06:44.723920 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:44.724426 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:44.724647 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:44.724962 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.307s (SENT) I0507 20:06:44.725077 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:44.725397 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.513s (election complete) I0507 20:06:44.726421 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:44.726951 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:44.904069 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.178s (SENT) I0507 20:06:44.904193 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:44.904563 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.545s (election complete) I0507 20:06:44.904716 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:44.988582 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:44.988714 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.660s (starting election) I0507 20:06:44.988818 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:44.989295 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:44.989459 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:44.989727 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.449s (SENT) I0507 20:06:44.989809 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:44.990089 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.357s (election complete) I0507 20:06:44.990175 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:44.990474 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:45.016465 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:45.016578 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.626s (starting election) I0507 20:06:45.016670 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:45.017104 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:45.017271 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:45.017522 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.618s (SENT) I0507 20:06:45.017596 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:45.017769 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.566s (election complete) I0507 20:06:45.017845 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:45.018266 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:45.098562 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.774s (SENT) I0507 20:06:45.098696 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:45.099169 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.855s (election complete) I0507 20:06:45.099298 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:45.450911 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:45.451055 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.545s (starting election) I0507 20:06:45.451174 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:45.451683 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:45.451894 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:45.452903 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:45.584684 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:45.584861 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.414s (starting election) I0507 20:06:45.584985 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:45.585559 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:45.585851 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:45.586956 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:45.643911 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.626s (SENT) I0507 20:06:45.644065 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:45.644474 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.189s (election complete) I0507 20:06:45.644610 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:45.650797 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.660s (SENT) I0507 20:06:45.650928 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:45.651269 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.986s (election complete) I0507 20:06:45.651361 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:45.834038 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:45.834206 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.316s (starting election) I0507 20:06:45.834316 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:45.834910 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:45.835175 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:45.836124 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:45.996981 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:45.997115 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.506s (starting election) I0507 20:06:45.997231 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:45.997704 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:45.997930 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:45.998237 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.545s (SENT) I0507 20:06:45.998327 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:45.998551 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.189s (election complete) I0507 20:06:45.998620 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:45.998852 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:45.999963 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.414s (SENT) I0507 20:06:46.000030 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:46.000239 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.220s (election complete) I0507 20:06:46.000327 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:46.152269 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.316s (SENT) I0507 20:06:46.152395 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:46.152859 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.732s (election complete) I0507 20:06:46.153014 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:46.188679 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:46.188839 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.412s (starting election) I0507 20:06:46.188949 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:46.189451 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:46.189637 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:46.190426 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:46.505259 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.506s (SENT) I0507 20:06:46.505389 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:46.505854 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.948s (election complete) I0507 20:06:46.505977 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:46.602373 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.412s (SENT) I0507 20:06:46.602506 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:46.602972 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.935s (election complete) I0507 20:06:46.603126 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:46.638633 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:46.638837 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.695s (starting election) I0507 20:06:46.639021 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:46.639587 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:46.639827 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:46.640813 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:46.886296 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:46.886454 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.372s (starting election) I0507 20:06:46.886561 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:46.887133 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:46.887343 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:46.888294 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:47.259430 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:47.259553 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.944s (starting election) I0507 20:06:47.259665 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:47.260123 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:47.260318 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:47.261423 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.372s (SENT) I0507 20:06:47.261534 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:47.261854 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.162s (election complete) I0507 20:06:47.261955 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:47.262110 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:47.335245 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:47.335412 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.804s (starting election) I0507 20:06:47.335517 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:47.336045 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:47.336228 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:47.336549 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.695s (SENT) I0507 20:06:47.336663 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:47.337024 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.633s (election complete) I0507 20:06:47.337112 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:47.337204 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:47.424957 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:47.425112 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.994s (starting election) I0507 20:06:47.425251 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:47.425876 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:47.426126 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:47.427265 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:47.515759 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:44.514441 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:44.514796 (+ 355us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:44.514817 (+ 21us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:44.514991 (+ 174us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:47.515353 (+3000362us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":109} W0507 20:06:47.516103 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:47.538698 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:47.538892 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.907s (starting election) I0507 20:06:47.538990 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:47.539537 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:47.539746 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:47.540827 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:47.971500 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:47.971657 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.776s (starting election) I0507 20:06:47.971762 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:47.972262 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:47.972486 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:47.973466 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:48.141759 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.804s (SENT) I0507 20:06:48.141883 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:48.142299 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.290s (election complete) I0507 20:06:48.142458 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:48.205220 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.944s (SENT) I0507 20:06:48.205346 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:48.205704 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.692s (election complete) I0507 20:06:48.205855 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:48.421654 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.994s (SENT) I0507 20:06:48.421780 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:48.422142 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.246s (election complete) I0507 20:06:48.422271 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:48.432842 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:48.432994 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.972s (starting election) I0507 20:06:48.433094 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:48.433600 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:48.433856 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:48.434828 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:48.446588 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:48.446708 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.526s (starting election) I0507 20:06:48.446825 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:48.447331 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:48.447571 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:48.447995 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.907s (SENT) I0507 20:06:48.448093 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:06:48.448374 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:48.448416 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.628s (election complete) I0507 20:06:48.448516 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:48.669260 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:48.669416 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.923s (starting election) I0507 20:06:48.669533 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:48.670043 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:48.670253 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:48.671303 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:48.750340 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.776s (SENT) I0507 20:06:48.750464 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:48.750859 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.658s (election complete) I0507 20:06:48.750965 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:48.974462 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.526s (SENT) I0507 20:06:48.974566 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:48.974954 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.179s (election complete) I0507 20:06:48.975065 1003 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:49.155092 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:49.155221 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.850s (starting election) I0507 20:06:49.155321 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:49.155819 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:49.156025 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:49.157034 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:49.407716 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.972s (SENT) I0507 20:06:49.407833 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:49.408267 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.264s (election complete) I0507 20:06:49.408417 1003 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:49.593292 1003 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:49.593505 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.611s (starting election) I0507 20:06:49.593626 1003 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:49.594130 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:49.594362 1003 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:49.595573 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.923s (SENT) I0507 20:06:49.595677 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:06:49.595835 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:49.595968 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.850s (election complete) I0507 20:06:49.596032 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:49.673401 1003 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:49.673535 1003 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.823s (starting election) I0507 20:06:49.673667 1003 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:49.674263 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:49.674517 1003 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:49.675324 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:50.006325 1003 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:50.006553 1003 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.916s (starting election) I0507 20:06:50.006695 1003 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:50.007091 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.850s (SENT) I0507 20:06:50.007187 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:50.007582 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:50.007908 1003 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:50.007917 1010 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.198s (election complete) I0507 20:06:50.008013 1010 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:50.008626 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:50.206547 1010 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:50.206697 1010 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.627s (starting election) I0507 20:06:50.206810 1010 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:50.207190 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.611s (SENT) I0507 20:06:50.207329 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:50.207471 1010 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:50.207660 1003 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.778s (election complete) I0507 20:06:50.207751 1010 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:50.207749 1003 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:50.208585 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:50.497270 1010 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:50.497470 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.176s (starting election) I0507 20:06:50.497632 1010 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:50.498302 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:50.498560 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:50.498749 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.823s (SENT) I0507 20:06:50.498914 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:50.499263 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.138s (election complete) I0507 20:06:50.499397 1010 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:50.499795 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:50.638701 1010 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:50.638865 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (starting election) I0507 20:06:50.639009 1010 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:50.639559 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:50.639772 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:50.640918 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:50.675942 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.176s (SENT) I0507 20:06:50.676072 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:50.676518 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.583s (election complete) I0507 20:06:50.676717 1010 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:50.739718 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:47.741683 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:47.741988 (+ 305us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:47.742004 (+ 16us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:47.742133 (+ 129us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:50.739289 (+2997156us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":87} W0507 20:06:50.740022 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:50.834987 1010 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:50.835153 1010 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.377s (starting election) I0507 20:06:50.835248 1010 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:50.835759 1010 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:50.836820 1010 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:50.836892 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.627s (SENT) I0507 20:06:50.837056 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:50.837352 1010 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.256s (election complete) I0507 20:06:50.837445 1010 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:50.837837 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:50.926102 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.916s (SENT) I0507 20:06:50.926218 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:50.926672 1010 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.207s (election complete) I0507 20:06:50.926803 1010 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:50.962208 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.321s (SENT) I0507 20:06:50.962345 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:50.962643 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.541s (election complete) I0507 20:06:50.962700 1010 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:50.987185 1010 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:50.987332 1010 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.608s (starting election) I0507 20:06:50.987475 1010 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:50.987965 1010 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:50.988185 1010 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:50.989073 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:51.134238 1010 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:51.135212 1010 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.634s (starting election) I0507 20:06:51.135332 1010 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:51.135779 1010 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:51.135988 1010 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:51.136989 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:51.215754 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.377s (SENT) I0507 20:06:51.215890 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:51.216341 1010 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.746s (election complete) I0507 20:06:51.216460 1010 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:51.504142 1010 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:51.504295 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.101s (starting election) I0507 20:06:51.504446 1010 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:51.505082 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:51.505342 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:51.506233 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:51.596709 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.608s (SENT) I0507 20:06:51.596711 1010 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:51.596827 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:51.596884 1010 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.678s (starting election) I0507 20:06:51.596989 1010 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:51.597617 1010 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:51.597625 1013 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.778s (election complete) I0507 20:06:51.597692 1013 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:51.597818 1010 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:51.598722 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:51.606220 1010 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:51.606387 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.879s (starting election) I0507 20:06:51.606505 1010 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:51.607023 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:51.607270 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:51.607645 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.101s (SENT) I0507 20:06:51.607717 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:51.607937 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.098s (election complete) I0507 20:06:51.608029 1010 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:51.608237 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:51.707211 1010 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:51.707352 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.294s (starting election) I0507 20:06:51.707443 1010 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:51.708801 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:51.709005 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:51.709816 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:51.771270 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.634s (SENT) I0507 20:06:51.771378 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:51.771787 1010 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.387s (election complete) I0507 20:06:51.771900 1010 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:52.002769 1010 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:52.002924 1010 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.416s (starting election) I0507 20:06:52.003036 1010 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:52.003513 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.294s (SENT) I0507 20:06:52.003568 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:52.003641 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:52.003861 1010 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:52.003973 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.483s (election complete) I0507 20:06:52.004051 1013 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:52.004767 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:52.160184 1013 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:52.160326 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.270s (starting election) I0507 20:06:52.160439 1013 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:52.160997 1013 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:52.161229 1013 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:52.162140 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:52.277670 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.678s (SENT) I0507 20:06:52.277802 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:52.278235 1013 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.902s (election complete) I0507 20:06:52.279337 1013 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:52.421474 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.416s (SENT) I0507 20:06:52.421588 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:52.421919 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.500s (election complete) I0507 20:06:52.422083 1013 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:52.431090 1013 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:52.431255 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.475s (starting election) I0507 20:06:52.431397 1013 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:52.431908 1013 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:52.432116 1013 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:52.432513 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.270s (SENT) I0507 20:06:52.432595 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:52.432893 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.944s (election complete) I0507 20:06:52.432988 1013 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:52.433101 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:52.487546 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.879s (SENT) I0507 20:06:52.487675 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:52.488051 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.297s (election complete) I0507 20:06:52.488162 1013 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:52.786672 1013 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:52.786851 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.174s (starting election) I0507 20:06:52.787055 1013 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:52.787703 1013 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:52.788007 1013 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:52.789042 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:52.908495 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.475s (SENT) I0507 20:06:52.908602 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:52.909049 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.173s (election complete) I0507 20:06:52.909214 1013 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:52.962386 1013 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:52.962569 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.803s (starting election) I0507 20:06:52.962697 1013 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:52.963299 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.174s (SENT) I0507 20:06:52.963399 1013 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:52.963414 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:52.963652 1013 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:52.963851 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.884s (election complete) I0507 20:06:52.963913 1013 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:52.964540 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:53.083269 1014 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:53.083446 1014 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.082s (starting election) I0507 20:06:53.083576 1014 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:53.084218 1014 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:53.084475 1014 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:53.085386 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:53.166136 1014 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:53.166298 1014 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.564s (starting election) I0507 20:06:53.166409 1014 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:53.167098 1014 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:53.167157 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.082s (SENT) I0507 20:06:53.167228 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:53.167327 1014 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:53.167467 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.380s (election complete) I0507 20:06:53.167542 1013 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:53.168165 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:53.181593 1013 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:53.181717 1013 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.113s (starting election) I0507 20:06:53.181823 1013 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:53.182320 1013 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:53.182543 1013 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:53.183605 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:53.295606 1013 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:53.295761 1013 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.969s (starting election) I0507 20:06:53.295878 1013 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:53.296519 1013 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:53.296643 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.113s (SENT) I0507 20:06:53.296761 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:53.296772 1013 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:53.297070 1013 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.746s (election complete) I0507 20:06:53.297158 1013 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:53.297787 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:53.548506 1013 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:53.548629 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.618s (starting election) I0507 20:06:53.548727 1013 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:53.549204 1013 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:53.549412 1013 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:53.550487 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:53.733400 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.564s (SENT) I0507 20:06:53.733515 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:53.733956 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.236s (election complete) I0507 20:06:53.734092 1013 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:53.767962 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.803s (SENT) I0507 20:06:53.768086 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:53.768445 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.338s (election complete) I0507 20:06:53.768543 1013 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:53.971465 1013 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:53.971665 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.206s (starting election) I0507 20:06:53.971829 1013 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:53.972390 1013 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:53.972584 1013 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:53.973662 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:53.975646 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:50.970612 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:50.970926 (+ 314us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:50.970942 (+ 16us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:50.971040 (+ 98us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:53.975265 (+3004225us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":104} W0507 20:06:53.975890 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:54.044078 1013 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:54.044240 1013 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.234s (starting election) I0507 20:06:54.045271 1013 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:54.045943 1013 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:54.046203 1013 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:54.047500 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:54.107403 1013 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:54.107571 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.219s (starting election) I0507 20:06:54.107686 1013 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:54.108278 1013 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:54.108497 1013 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:54.109683 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:54.168916 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.618s (SENT) I0507 20:06:54.169044 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:54.169443 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.172s (election complete) I0507 20:06:54.169541 1013 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:54.179348 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.206s (SENT) I0507 20:06:54.179452 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:54.179811 1013 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.699s (election complete) I0507 20:06:54.179921 1013 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:54.268523 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.969s (SENT) I0507 20:06:54.268645 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:54.269027 1013 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.236s (election complete) I0507 20:06:54.269145 1013 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:54.280587 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.234s (SENT) I0507 20:06:54.280740 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:54.281172 1013 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.619s (election complete) I0507 20:06:54.281271 1013 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:54.327237 1013 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:54.327392 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.569s (starting election) I0507 20:06:54.327553 1013 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:54.328171 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.219s (SENT) I0507 20:06:54.328183 1013 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:54.328250 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:54.328475 1013 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:54.328646 1013 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.547s (election complete) I0507 20:06:54.328769 1013 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:54.329558 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:54.876595 1018 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:54.876715 1018 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.999s (starting election) I0507 20:06:54.876915 1018 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:54.877348 1018 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:54.877507 1018 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:54.878567 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:54.880084 1018 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:54.880165 1018 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.418s (starting election) I0507 20:06:54.880228 1018 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:54.880548 1018 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:54.880663 1018 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:54.881319 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:54.899601 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.569s (SENT) I0507 20:06:54.899709 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:54.900133 1018 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.453s (election complete) I0507 20:06:54.900257 1018 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:54.901269 1018 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:54.901345 1018 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.111s (starting election) I0507 20:06:54.901437 1018 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:54.901933 1018 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:54.902158 1018 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:54.903146 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:55.012958 1018 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:55.013139 1018 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.908s (starting election) I0507 20:06:55.013288 1018 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } W0507 20:06:55.014621 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.111s (SENT) I0507 20:06:55.014693 1018 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:55.014775 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:55.015038 1018 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:55.015274 1018 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.693s (election complete) I0507 20:06:55.015373 1018 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:55.016016 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:55.299089 1018 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:55.299221 1018 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.138s (starting election) I0507 20:06:55.299373 1018 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:55.299592 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.418s (SENT) I0507 20:06:55.299693 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:55.299994 1018 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:55.300007 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.202s (election complete) I0507 20:06:55.300153 1020 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:55.300245 1018 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:55.300925 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:55.354079 1018 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:55.354262 1018 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.187s (starting election) I0507 20:06:55.354377 1018 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:55.354925 1018 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:55.355123 1018 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:55.356111 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:55.439429 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.138s (SENT) I0507 20:06:55.439538 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:55.439934 1018 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.611s (election complete) I0507 20:06:55.440099 1018 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:06:55.542618 1018 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:55.542753 1018 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.989s (starting election) I0507 20:06:55.542898 1018 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:06:55.543344 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.187s (SENT) I0507 20:06:55.543426 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:55.543454 1018 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:55.543665 1018 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:55.543803 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.421s (election complete) I0507 20:06:55.543902 1020 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:55.545404 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:55.709297 1020 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:55.709451 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.277s (starting election) I0507 20:06:55.709594 1020 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:55.710219 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:55.710496 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:55.711653 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:55.878654 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.999s (SENT) I0507 20:06:55.878782 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:55.879160 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.078s (election complete) I0507 20:06:55.879510 1020 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:55.925163 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.908s (SENT) I0507 20:06:55.925276 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:55.925616 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.741s (election complete) I0507 20:06:55.925725 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:55.958087 1020 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:55.958308 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.896s (starting election) I0507 20:06:55.958447 1020 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:55.959192 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:55.959483 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:55.960624 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:55.988639 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.277s (SENT) I0507 20:06:55.988811 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:55.989302 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.360s (election complete) I0507 20:06:55.989428 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:56.052292 1020 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:56.052415 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.896s (starting election) I0507 20:06:56.052518 1020 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:56.052968 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:56.053930 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:56.054172 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:56.350366 1020 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:56.350527 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.930s (starting election) I0507 20:06:56.350637 1020 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:56.351161 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:56.351398 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:56.352355 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:56.534595 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.989s (SENT) I0507 20:06:56.534705 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:56.535145 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.804s (election complete) I0507 20:06:56.535284 1020 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:56.856662 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.896s (SENT) I0507 20:06:56.856818 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:56.857264 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.939s (election complete) I0507 20:06:56.857381 1020 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:56.949908 1020 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:56.950027 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.938s (starting election) I0507 20:06:56.950153 1020 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:56.950613 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:56.950784 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:56.951048 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.896s (SENT) I0507 20:06:56.951119 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:56.951478 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.230s (election complete) I0507 20:06:56.951639 1020 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:56.952330 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:57.182013 1020 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:57.182147 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.854s (starting election) I0507 20:06:57.182240 1020 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:57.182689 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:57.182857 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:57.183784 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:57.211510 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:54.213643 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:54.213956 (+ 313us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:54.213974 (+ 18us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:54.214095 (+ 121us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:06:57.211190 (+2997095us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":74} W0507 20:06:57.211751 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:06:57.281716 1020 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:57.281833 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.261s (starting election) I0507 20:06:57.281951 1020 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:57.282407 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:57.282618 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:57.282908 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.930s (SENT) I0507 20:06:57.282984 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:57.283242 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.103s (election complete) I0507 20:06:57.283378 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:57.283756 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:57.387022 1020 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:57.387162 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.661s (starting election) I0507 20:06:57.387251 1020 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:57.387717 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:57.387926 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:57.389107 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:57.544433 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.261s (SENT) I0507 20:06:57.544535 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:57.544848 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.334s (election complete) I0507 20:06:57.544946 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:57.797384 1020 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:57.797492 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.809s (starting election) I0507 20:06:57.797592 1020 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:57.798017 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:57.798204 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:57.799151 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:57.879915 1020 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:57.880049 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.641s (starting election) I0507 20:06:57.880136 1020 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:57.880594 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:57.880762 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:57.881778 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:57.890569 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.938s (SENT) I0507 20:06:57.890681 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:57.891113 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.429s (election complete) I0507 20:06:57.891212 1020 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:58.037871 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.854s (SENT) I0507 20:06:58.037983 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:58.038393 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.506s (election complete) I0507 20:06:58.038520 1020 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:58.050691 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.661s (SENT) I0507 20:06:58.050801 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:58.051142 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.292s (election complete) I0507 20:06:58.051260 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:58.343899 1020 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:58.344007 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.950s (starting election) I0507 20:06:58.344099 1020 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:58.344521 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:58.344663 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:58.345369 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:58.522754 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.641s (SENT) I0507 20:06:58.522902 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:58.523283 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.723s (election complete) I0507 20:06:58.523391 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:58.544970 1020 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:58.545179 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.803s (starting election) I0507 20:06:58.545284 1020 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:58.545795 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:58.545989 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:58.546918 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:58.607578 1020 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:58.607722 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.988s (starting election) I0507 20:06:58.607853 1020 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:58.608378 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:58.608515 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.809s (SENT) I0507 20:06:58.608613 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:58.608652 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:58.608939 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.316s (election complete) I0507 20:06:58.609036 1020 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:58.609657 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:58.926128 1020 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:58.926300 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.627s (starting election) I0507 20:06:58.926401 1020 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:58.926935 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:58.927135 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:58.927881 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:59.247037 1020 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:59.247159 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (starting election) I0507 20:06:59.247267 1020 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:59.247736 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:59.247936 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:59.248821 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:59.295428 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.950s (SENT) I0507 20:06:59.295549 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:59.295874 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.398s (election complete) I0507 20:06:59.295980 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:59.349098 1020 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:59.349232 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.896s (starting election) I0507 20:06:59.349330 1020 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:59.349798 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:59.350008 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:59.350015 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.803s (SENT) I0507 20:06:59.350090 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:06:59.350284 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (election complete) I0507 20:06:59.350335 1020 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:06:59.350726 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:59.553923 1020 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:59.554039 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.765s (starting election) I0507 20:06:59.554138 1020 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:59.554603 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:59.554798 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:59.555084 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.627s (SENT) I0507 20:06:59.555145 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:06:59.555550 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:59.555728 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.390s (election complete) I0507 20:06:59.555822 1020 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:06:59.598354 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.988s (SENT) I0507 20:06:59.598462 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:59.598774 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.357s (election complete) I0507 20:06:59.598897 1020 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:06:59.694722 1020 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:59.695650 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.810s (starting election) I0507 20:06:59.695757 1020 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:06:59.696213 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:06:59.696357 1020 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:06:59.697124 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:06:59.956259 1020 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:06:59.956374 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (starting election) I0507 20:06:59.956472 1020 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:06:59.956941 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:06:59.957159 1020 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:06:59.958027 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:06:59.966042 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.716s (SENT) I0507 20:06:59.966146 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:06:59.966469 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.599s (election complete) I0507 20:06:59.966568 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:00.214536 1020 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:00.214717 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.556s (starting election) I0507 20:07:00.214844 1020 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:00.216249 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:00.216449 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:00.217272 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:00.248363 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.896s (SENT) I0507 20:07:00.248451 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:00.248723 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.283s (election complete) I0507 20:07:00.248805 1020 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:00.320598 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.765s (SENT) I0507 20:07:00.320719 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:00.321027 1020 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.743s (election complete) I0507 20:07:00.321105 1020 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:00.455451 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:06:57.456585 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:06:57.456895 (+ 310us) negotiation.cc:98] Waiting for socket to connect 0507 20:06:57.456910 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:06:57.457008 (+ 98us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:00.455164 (+2998156us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":105} W0507 20:07:00.455724 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:07:00.507406 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.810s (SENT) I0507 20:07:00.507555 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:00.507962 1020 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.887s (election complete) I0507 20:07:00.508077 1020 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:00.533107 1020 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:00.533224 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.167s (starting election) I0507 20:07:00.533318 1020 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:00.533762 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:00.533943 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:00.534651 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:00.701278 1020 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:00.701395 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.410s (starting election) W0507 20:07:00.701462 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.167s (SENT) I0507 20:07:00.701536 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:00.701475 1020 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:00.702044 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:00.702178 1020 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:00.702343 1020 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.802s (election complete) I0507 20:07:00.702497 1020 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:00.702898 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:00.773224 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.556s (SENT) I0507 20:07:00.773327 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:00.773605 1026 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.797s (election complete) I0507 20:07:00.773716 1026 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:00.821553 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.863s (SENT) I0507 20:07:00.821699 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:00.822031 1026 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.594s (election complete) I0507 20:07:00.822129 1026 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:01.113962 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.410s (SENT) I0507 20:07:01.114084 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:01.114466 1026 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.797s (election complete) I0507 20:07:01.114544 1026 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:01.395812 1026 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:01.395925 1026 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.194s (starting election) I0507 20:07:01.396021 1026 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:01.396455 1026 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:01.396654 1026 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:01.397815 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:01.416887 1026 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:01.417032 1026 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.517s (starting election) I0507 20:07:01.417142 1026 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:01.417655 1026 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:01.417855 1026 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:01.418665 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:01.590477 1026 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:01.590620 1026 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.926s (starting election) I0507 20:07:01.590737 1026 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:01.591352 1026 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:01.591604 1026 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:01.591688 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.194s (SENT) I0507 20:07:01.591831 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:01.592082 1026 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.365s (election complete) I0507 20:07:01.592160 1026 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:01.592587 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:01.912506 1026 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:01.912686 1026 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.438s (starting election) I0507 20:07:01.912780 1026 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:01.913247 1026 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:01.913484 1026 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:01.914261 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:01.934871 1026 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:01.935039 1026 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.697s (starting election) I0507 20:07:01.935164 1026 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:01.935676 1026 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:01.935847 1026 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:01.936120 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.517s (SENT) I0507 20:07:01.936182 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:01.936450 1026 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.593s (election complete) I0507 20:07:01.936555 1026 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:01.936615 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:01.958856 1026 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:01.959002 1026 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.624s (starting election) I0507 20:07:01.959096 1026 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:01.959591 1026 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:01.959751 1026 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:01.960762 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:02.351567 1026 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:02.351689 1026 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.885s (starting election) I0507 20:07:02.351792 1026 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:02.352291 1026 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:02.352368 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.438s (SENT) I0507 20:07:02.352445 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:02.352988 1026 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:02.353205 1026 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.352s (election complete) I0507 20:07:02.353328 1026 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:02.353484 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:02.519292 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.926s (SENT) I0507 20:07:02.519421 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:02.519788 1028 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.800s (election complete) I0507 20:07:02.519935 1028 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:02.530828 1028 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:02.530967 1028 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.740s (starting election) I0507 20:07:02.531074 1028 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:02.531617 1028 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:02.531829 1028 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:02.532666 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:02.584753 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.624s (SENT) I0507 20:07:02.584864 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:02.585172 1028 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.714s (election complete) I0507 20:07:02.585324 1028 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:02.634749 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.697s (SENT) I0507 20:07:02.634871 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:02.635205 1028 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.345s (election complete) I0507 20:07:02.635339 1028 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:02.706369 1028 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:02.706526 1028 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.085s (starting election) I0507 20:07:02.706624 1028 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:02.707170 1028 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:02.707330 1028 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:02.708137 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:02.792335 1028 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:02.792441 1028 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.788s (starting election) I0507 20:07:02.792590 1028 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:02.792752 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.085s (SENT) I0507 20:07:02.792846 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:02.793031 1028 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:02.793045 1026 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.884s (election complete) I0507 20:07:02.793114 1026 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:02.793604 1028 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:02.793830 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:02.981382 1028 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:02.981567 1028 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.224s (starting election) I0507 20:07:02.981672 1028 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:02.982141 1028 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:02.982319 1028 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:02.983175 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:03.207010 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.224s (SENT) I0507 20:07:03.207125 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:03.207094 1028 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:03.207226 1028 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.894s (starting election) I0507 20:07:03.207331 1028 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:03.207885 1028 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:03.207939 1026 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.996s (election complete) I0507 20:07:03.208057 1026 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:03.208096 1028 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:03.208932 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:03.240594 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.885s (SENT) I0507 20:07:03.240718 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:03.241077 1028 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.698s (election complete) I0507 20:07:03.241194 1028 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:03.273512 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.740s (SENT) I0507 20:07:03.273617 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:03.273939 1028 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.985s (election complete) I0507 20:07:03.274073 1028 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:03.299872 1028 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:03.300016 1028 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.837s (starting election) I0507 20:07:03.300119 1028 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:03.300586 1028 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:03.300770 1028 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:03.301757 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:03.583577 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.788s (SENT) I0507 20:07:03.583689 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:03.584049 1028 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.510s (election complete) I0507 20:07:03.584146 1028 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:03.703542 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:00.705541 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:00.705733 (+ 192us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:00.705744 (+ 11us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:00.705823 (+ 79us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:03.703245 (+2997422us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":57} W0507 20:07:03.703794 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:04.094967 1031 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:04.095072 1031 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.796s (starting election) I0507 20:07:04.095173 1031 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:04.095613 1031 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:04.095783 1031 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:04.096629 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:04.103566 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.894s (SENT) I0507 20:07:04.103655 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:04.103947 1031 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.643s (election complete) I0507 20:07:04.104053 1031 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:04.137586 1031 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:04.137754 1031 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.501s (starting election) I0507 20:07:04.137862 1031 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:04.138425 1031 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:04.138622 1031 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:04.138715 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.837s (SENT) I0507 20:07:04.138820 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:04.139122 1031 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.460s (election complete) I0507 20:07:04.139245 1031 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:04.139617 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:04.600198 1031 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:04.600316 1031 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.301s (starting election) I0507 20:07:04.600435 1031 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:04.600883 1031 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:04.601068 1031 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:04.601975 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:04.642001 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.501s (SENT) I0507 20:07:04.642107 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:04.642429 1031 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.208s (election complete) I0507 20:07:04.642524 1031 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:04.747822 1031 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:04.747952 1031 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.122s (starting election) I0507 20:07:04.748044 1031 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:04.748514 1031 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:04.748688 1031 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:04.749441 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:04.851784 1031 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:04.851900 1031 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.846s (starting election) I0507 20:07:04.851992 1031 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:04.852457 1031 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:04.852635 1031 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:04.853408 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:04.871032 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.122s (SENT) I0507 20:07:04.871022 1031 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:04.871129 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:04.871161 1031 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.214s (starting election) I0507 20:07:04.871243 1031 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:04.871695 1031 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:04.871877 1031 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:04.872028 1031 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.536s (election complete) I0507 20:07:04.872108 1031 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:04.872762 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:04.891518 1033 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:04.891628 1033 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.664s (starting election) I0507 20:07:04.891763 1033 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:04.892300 1033 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:04.892460 1033 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:04.892750 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.796s (SENT) I0507 20:07:04.892876 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:04.893030 1033 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.494s (election complete) W0507 20:07:04.893085 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:04.893134 1033 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:04.903079 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.301s (SENT) I0507 20:07:04.903162 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:04.903435 1033 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.735s (election complete) I0507 20:07:04.903534 1033 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:05.086906 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.214s (SENT) I0507 20:07:05.086997 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:05.088091 1033 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.112s (election complete) I0507 20:07:05.088208 1033 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:05.201642 1033 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:05.201769 1033 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.131s (starting election) I0507 20:07:05.201858 1033 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:05.202301 1033 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:05.202458 1033 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:05.203176 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:05.333956 1033 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:05.333961 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.131s (SENT) I0507 20:07:05.334094 1033 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.671s (starting election) I0507 20:07:05.334142 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:05.334197 1033 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:05.334589 1033 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:05.334651 1031 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.870s (election complete) I0507 20:07:05.334717 1031 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:05.334792 1033 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:05.335467 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:05.387895 1033 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:05.388051 1033 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.631s (starting election) I0507 20:07:05.388175 1033 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:05.388756 1033 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:05.388944 1033 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:05.389678 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:05.557505 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.664s (SENT) I0507 20:07:05.557663 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:05.557974 1033 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.847s (election complete) I0507 20:07:05.558071 1033 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:05.638955 1033 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:05.639053 1033 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.176s (starting election) I0507 20:07:05.639144 1033 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:05.639551 1033 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:05.639694 1033 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:05.640455 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:05.699782 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.846s (SENT) I0507 20:07:05.699884 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:05.700191 1033 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.544s (election complete) I0507 20:07:05.700285 1033 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:05.816849 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.176s (SENT) I0507 20:07:05.816956 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:05.817297 1033 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.225s (election complete) I0507 20:07:05.817399 1033 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:06.007884 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.671s (SENT) I0507 20:07:06.007978 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:06.008304 1033 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.968s (election complete) I0507 20:07:06.008396 1033 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:06.022174 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.631s (SENT) I0507 20:07:06.022266 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:06.022523 1033 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.912s (election complete) I0507 20:07:06.022605 1033 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:06.043309 1033 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:06.043442 1033 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.125s (starting election) I0507 20:07:06.043536 1033 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:06.043980 1033 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:06.044149 1033 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:06.045033 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:06.169795 1033 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:06.169883 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.125s (SENT) I0507 20:07:06.170734 1033 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.894s (starting election) I0507 20:07:06.170737 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:06.170856 1033 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:06.171308 1033 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:06.171492 1033 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:06.171617 1033 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.591s (election complete) I0507 20:07:06.171738 1033 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:06.172313 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:06.764053 1035 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:06.764156 1035 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.215s (starting election) I0507 20:07:06.764282 1035 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:06.764693 1035 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:06.764873 1035 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:06.765837 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:06.935747 1035 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:06.935880 1035 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.479s (starting election) I0507 20:07:06.935958 1035 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:06.936354 1035 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:06.936520 1035 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:06.937512 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:06.971489 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:03.971652 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:03.971948 (+ 296us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:03.971967 (+ 19us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:03.972093 (+ 126us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:06.971198 (+2999105us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":84} W0507 20:07:06.971808 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:06.976871 1035 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:06.976984 1035 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.141s (starting election) I0507 20:07:06.977082 1035 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:06.977519 1035 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:06.977721 1035 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:06.978651 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:06.980453 1035 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:06.980520 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.215s (SENT) I0507 20:07:06.980573 1035 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.983s (starting election) I0507 20:07:06.980589 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:06.980646 1035 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:06.980964 1035 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:06.981087 1035 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:06.981225 1035 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.895s (election complete) I0507 20:07:06.981271 1035 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:06.981906 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:07.066427 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.894s (SENT) I0507 20:07:07.066529 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:07.066824 1037 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.285s (election complete) I0507 20:07:07.066936 1037 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:07.119386 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.141s (SENT) I0507 20:07:07.119388 1037 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:07.119506 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:07.119509 1037 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.182s (starting election) I0507 20:07:07.119626 1037 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:07.120167 1037 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:07.120182 1035 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.915s (election complete) I0507 20:07:07.120249 1035 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:07.120388 1037 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:07.121281 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:07.304339 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.182s (SENT) I0507 20:07:07.304471 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:07.304814 1037 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.994s (election complete) I0507 20:07:07.304929 1037 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:07.352691 1037 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:07.352816 1037 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.510s (starting election) I0507 20:07:07.352910 1037 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:07.353345 1037 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:07.353505 1037 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:07.354487 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:07.415889 1037 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:07.416002 1037 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.348s (starting election) I0507 20:07:07.416090 1037 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:07.416517 1037 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:07.416640 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.479s (SENT) I0507 20:07:07.416705 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:07.416731 1037 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:07.416894 1037 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.896s (election complete) I0507 20:07:07.416949 1037 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:07.417623 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:07.765866 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.348s (SENT) I0507 20:07:07.765978 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:07.766345 1037 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.332s (election complete) I0507 20:07:07.766468 1037 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:07.863955 1037 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:07.864096 1037 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.437s (starting election) I0507 20:07:07.864193 1037 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:07.864656 1037 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:07.864867 1037 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:07.865227 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.510s (SENT) I0507 20:07:07.865336 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:07.865634 1037 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.269s (election complete) I0507 20:07:07.865702 1037 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:07.865684 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:07.965793 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.983s (SENT) I0507 20:07:07.965894 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:07.966200 1037 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.517s (election complete) I0507 20:07:07.966289 1037 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:08.099162 1037 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:08.099263 1037 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.699s (starting election) I0507 20:07:08.099351 1037 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:08.099750 1037 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:08.099906 1037 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:08.100960 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:08.299324 1037 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:08.299515 1037 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.571s (starting election) I0507 20:07:08.299643 1037 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:08.300281 1037 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:08.300549 1037 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:08.301345 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:08.302232 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.437s (SENT) I0507 20:07:08.302295 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:08.302570 1037 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.935s (election complete) I0507 20:07:08.302661 1037 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:08.799788 1037 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:08.799973 1037 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.766s (starting election) I0507 20:07:08.800168 1037 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:08.800923 1037 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:08.800921 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.699s (SENT) I0507 20:07:08.801065 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:08.801254 1037 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:08.801471 1037 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.881s (election complete) I0507 20:07:08.801563 1037 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:08.802175 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:08.871719 1040 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:08.871868 1040 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.787s (starting election) I0507 20:07:08.872083 1040 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:08.872730 1040 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:08.872977 1040 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:08.873656 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.571s (SENT) I0507 20:07:08.873950 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:07:08.874181 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:08.874220 1040 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.220s (election complete) I0507 20:07:08.874302 1040 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:09.095783 1040 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:09.095930 1040 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.715s (starting election) I0507 20:07:09.096046 1040 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:09.096499 1040 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:09.096683 1040 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:09.098289 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:09.238742 1040 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:09.238847 1040 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.980s (starting election) I0507 20:07:09.238981 1040 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:09.239444 1040 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:09.239610 1040 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:09.240399 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:09.569555 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.766s (SENT) I0507 20:07:09.569644 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:09.569913 1040 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.085s (election complete) I0507 20:07:09.570013 1040 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:09.655557 1040 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:09.655689 1040 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.608s (starting election) I0507 20:07:09.655787 1040 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:09.656220 1040 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:09.656379 1040 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:09.657102 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:09.663075 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.787s (SENT) I0507 20:07:09.663153 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:09.663424 1040 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.271s (election complete) I0507 20:07:09.663506 1040 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:09.813678 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.715s (SENT) I0507 20:07:09.813786 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:09.814040 1040 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.711s (election complete) I0507 20:07:09.814117 1040 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:10.220022 1040 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:10.220140 1040 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.207s (starting election) I0507 20:07:10.220230 1040 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:10.220690 1040 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:10.220883 1040 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:10.221117 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.980s (SENT) I0507 20:07:10.221185 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:10.221359 1040 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.408s (election complete) I0507 20:07:10.221427 1040 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:10.221969 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:10.243525 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:07.243187 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:07.243469 (+ 282us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:07.243485 (+ 16us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:07.243593 (+ 108us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:10.243202 (+2999609us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":89} W0507 20:07:10.243749 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:10.264475 1040 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:10.264621 1040 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.271s (starting election) I0507 20:07:10.264722 1040 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:10.265163 1040 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:10.265341 1040 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:10.266449 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.608s (SENT) I0507 20:07:10.266515 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:07:10.266577 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:10.266716 1040 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.195s (election complete) I0507 20:07:10.266819 1040 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:10.429116 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.207s (SENT) I0507 20:07:10.429235 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:10.429625 1040 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.881s (election complete) I0507 20:07:10.429699 1040 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:10.463312 1040 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:10.463426 1040 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.106s (starting election) I0507 20:07:10.463523 1040 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:10.463955 1040 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:10.464181 1040 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:10.464922 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:10.526320 1040 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:10.526432 1040 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.204s (starting election) I0507 20:07:10.526510 1040 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:10.527029 1040 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:10.527185 1040 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:10.527858 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:10.536906 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.271s (SENT) I0507 20:07:10.537009 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:10.537326 1040 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.888s (election complete) I0507 20:07:10.537416 1040 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:10.571166 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.106s (SENT) I0507 20:07:10.571269 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:10.571632 1040 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.927s (election complete) I0507 20:07:10.571719 1040 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:10.731009 1040 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:10.731127 1040 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.227s (starting election) I0507 20:07:10.731251 1040 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:10.731853 1040 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:10.731851 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.204s (SENT) I0507 20:07:10.731953 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:10.732082 1040 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:10.732338 1040 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.290s (election complete) I0507 20:07:10.732417 1040 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:10.732934 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:10.960278 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.227s (SENT) I0507 20:07:10.960399 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:10.960750 1044 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.450s (election complete) I0507 20:07:10.960876 1044 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:11.312006 1044 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:11.312155 1044 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.378s (starting election) I0507 20:07:11.312300 1044 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:11.312827 1044 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:11.313060 1044 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:11.314533 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:11.411916 1044 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:11.412086 1044 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.403s (starting election) I0507 20:07:11.412189 1044 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:11.412711 1044 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:11.412920 1044 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:11.413853 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:11.499119 1044 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:11.499243 1044 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.371s (starting election) I0507 20:07:11.499354 1044 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:11.499886 1044 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:11.500998 1044 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:11.501186 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:11.691077 1044 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:11.691241 1044 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.196s (starting election) I0507 20:07:11.691356 1044 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:11.691892 1044 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:11.692091 1044 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:11.692225 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.378s (SENT) I0507 20:07:11.692322 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:11.692535 1044 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.519s (election complete) I0507 20:07:11.692602 1044 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:11.693099 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:11.816637 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.403s (SENT) I0507 20:07:11.816645 1044 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:11.817021 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:11.817066 1044 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.155s (starting election) I0507 20:07:11.817198 1044 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:11.817857 1044 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:11.818078 1044 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:11.818260 1046 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.365s (election complete) I0507 20:07:11.818322 1046 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:11.818862 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:11.871206 1046 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:11.871327 1046 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.882s (starting election) I0507 20:07:11.871434 1046 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:11.871960 1046 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:11.872151 1046 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:11.872577 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.371s (SENT) I0507 20:07:11.872683 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:07:11.872843 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:11.873267 1046 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.490s (election complete) I0507 20:07:11.873350 1046 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:11.889559 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.196s (SENT) I0507 20:07:11.889673 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:11.889998 1046 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.765s (election complete) I0507 20:07:11.890131 1046 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:11.974575 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.155s (SENT) I0507 20:07:11.974699 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:11.975067 1046 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.135s (election complete) I0507 20:07:11.975216 1046 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:12.110715 1046 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:12.110872 1046 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.681s (starting election) I0507 20:07:12.110992 1046 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:12.111425 1046 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:12.111567 1046 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:12.112321 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:12.363965 1046 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:12.364082 1046 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.774s (starting election) I0507 20:07:12.364181 1046 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:12.364624 1046 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:12.364815 1046 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:12.365798 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:12.656293 1046 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:12.656437 1046 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.615s (starting election) I0507 20:07:12.656536 1046 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:12.657038 1046 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:12.657253 1046 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:12.658252 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:12.755728 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.882s (SENT) I0507 20:07:12.755846 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:12.756167 1046 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.447s (election complete) I0507 20:07:12.756268 1046 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:12.793273 1046 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:12.793395 1046 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.278s (starting election) W0507 20:07:12.793555 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.681s (SENT) I0507 20:07:12.793658 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:12.793536 1046 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:12.793999 1046 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:12.794193 1046 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:12.794286 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.439s (election complete) I0507 20:07:12.794402 1047 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:12.795207 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:13.073335 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.278s (SENT) I0507 20:07:13.073444 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:13.073738 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.217s (election complete) I0507 20:07:13.073971 1047 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:13.140421 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.774s (SENT) I0507 20:07:13.140521 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:13.140887 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.855s (election complete) I0507 20:07:13.140965 1047 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:13.271802 1047 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:13.271937 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.156s (starting election) I0507 20:07:13.272070 1047 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:13.272512 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:13.272689 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:13.273746 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:13.274541 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.615s (SENT) I0507 20:07:13.274612 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:13.274804 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.757s (election complete) I0507 20:07:13.274921 1047 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:13.291556 1047 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:13.291697 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.427s (starting election) I0507 20:07:13.291795 1047 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:13.292376 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:13.292647 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:13.293776 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:13.429680 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.156s (SENT) I0507 20:07:13.429807 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:13.430162 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.379s (election complete) I0507 20:07:13.430238 1047 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:13.527451 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:10.530325 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:10.530564 (+ 239us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:10.530576 (+ 12us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:10.530708 (+ 132us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:13.527198 (+2996490us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":74} W0507 20:07:13.527709 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:13.719528 1047 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:13.719633 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.313s (starting election) I0507 20:07:13.719750 1047 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:13.720963 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:13.721027 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.427s (SENT) I0507 20:07:13.721140 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:13.721176 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:13.721432 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.608s (election complete) I0507 20:07:13.721496 1047 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:13.722000 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:13.810029 1047 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:13.810144 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.740s (starting election) I0507 20:07:13.810235 1047 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:13.810655 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:13.810802 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:13.811560 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:13.882210 1047 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:13.882318 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.393s (starting election) I0507 20:07:13.882434 1047 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:13.882920 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:13.883095 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:13.883954 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:13.996393 1047 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:13.996522 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.243s (starting election) I0507 20:07:13.996620 1047 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:13.997068 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:13.997253 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:13.997946 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:14.035465 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.313s (SENT) I0507 20:07:14.035568 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:14.035907 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.429s (election complete) I0507 20:07:14.035987 1047 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:14.240437 1047 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:14.240510 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.243s (SENT) I0507 20:07:14.240555 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.466s (starting election) I0507 20:07:14.240592 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:14.240659 1047 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:14.241128 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:14.241138 1050 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.850s (election complete) I0507 20:07:14.241204 1050 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:14.241367 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:14.241951 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:14.275990 1047 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:14.276137 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.653s (starting election) I0507 20:07:14.276254 1047 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:14.276793 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:14.277014 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:14.277264 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.393s (SENT) I0507 20:07:14.277333 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:14.277577 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.213s (election complete) I0507 20:07:14.277642 1047 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:14.277879 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:14.465883 1047 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:14.466014 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.123s (starting election) I0507 20:07:14.466110 1047 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:14.466545 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:14.466734 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:14.467460 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:14.491521 1047 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:14.491667 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.169s (starting election) I0507 20:07:14.491776 1047 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:14.492234 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:14.492406 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:14.493177 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:14.551653 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.740s (SENT) I0507 20:07:14.551759 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:14.552084 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.834s (election complete) I0507 20:07:14.552182 1047 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:14.590142 1047 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:14.590255 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.847s (starting election) W0507 20:07:14.590308 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.123s (SENT) I0507 20:07:14.590415 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:14.590386 1047 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:14.590831 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:14.590901 1050 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.519s (election complete) I0507 20:07:14.590976 1050 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:14.591006 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:14.591825 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:14.662344 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.169s (SENT) I0507 20:07:14.662458 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:14.662808 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.310s (election complete) I0507 20:07:14.662955 1047 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:14.708155 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.466s (SENT) I0507 20:07:14.708252 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:14.708582 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.294s (election complete) I0507 20:07:14.708696 1047 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:14.922499 232 maintenance_manager.cc:304] P 476aed1601be4605b69e0622e3eeb7fc: Scheduling FlushMRSOp(00000000000000000000000000000000): perf score=0.033367 I0507 20:07:14.923892 231 tablet.cc:1152] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Flush: entering stage 1 (old memrowset already frozen for inserts) I0507 20:07:14.923952 231 compaction.cc:913] Selected 1 rowsets to compact: I0507 20:07:14.923995 231 compaction.cc:916] memrowset(current size on disk: ~0 bytes) I0507 20:07:14.924019 231 tablet.cc:1154] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Memstore in-memory size: 7909 bytes I0507 20:07:14.924062 231 tablet.cc:1443] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6249363551657701376 or (T in {6249363551657701376})}] I0507 20:07:14.930258 231 multi_column_writer.cc:98] Opened CFile writers for 3 column(s) W0507 20:07:14.930899 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.653s (SENT) I0507 20:07:14.930985 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:14.931301 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.818s (election complete) I0507 20:07:14.931393 1047 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:14.942577 231 tablet.cc:1528] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I0507 20:07:14.943122 231 tablet.cc:1583] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Flush Phase 2: carrying over any updates which arrived during Phase 1 I0507 20:07:14.943194 231 tablet.cc:1585] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6249363551657701376 or (T in {6249363551657701376})}] I0507 20:07:14.948113 231 tablet.cc:1633] T 00000000000000000000000000000000 P 476aed1601be4605b69e0622e3eeb7fc: Flush successful on 6 rows (7518 bytes) I0507 20:07:14.948684 231 maintenance_manager.cc:494] P 476aed1601be4605b69e0622e3eeb7fc: FlushMRSOp(00000000000000000000000000000000) complete. Timing: real 0.025s user 0.024s sys 0.000s Metrics: {"cfile_init":1,"data dirs.queue_time_us":365,"data dirs.run_cpu_time_us":232,"data dirs.run_wall_time_us":1090,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4,"lbm_write_time_us":457,"lbm_writes_lt_1ms":27,"thread_start_us":360,"threads_started":2,"wal-append.queue_time_us":324} I0507 20:07:14.950422 232 maintenance_manager.cc:304] P 476aed1601be4605b69e0622e3eeb7fc: Scheduling UndoDeltaBlockGCOp(00000000000000000000000000000000): 766 bytes on disk I0507 20:07:14.951251 231 maintenance_manager.cc:494] P 476aed1601be4605b69e0622e3eeb7fc: UndoDeltaBlockGCOp(00000000000000000000000000000000) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":78,"lbm_reads_lt_1ms":4} I0507 20:07:15.003196 1047 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:15.003345 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.435s (starting election) I0507 20:07:15.003448 1047 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:15.003999 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:15.004168 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:15.005136 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:15.111084 1047 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:15.111203 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.415s (starting election) I0507 20:07:15.111310 1047 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:15.111747 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:15.111985 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:15.112813 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:15.439251 1047 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:15.439297 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.847s (SENT) I0507 20:07:15.439375 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.385s (starting election) I0507 20:07:15.439394 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:15.439469 1047 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:15.440930 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:15.440942 1053 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.441s (election complete) I0507 20:07:15.441022 1053 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:15.441084 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.435s (SENT) I0507 20:07:15.441145 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:15.441143 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:15.441351 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.948s (election complete) I0507 20:07:15.441418 1047 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:15.441961 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:15.527850 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.415s (SENT) I0507 20:07:15.527956 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:15.528246 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.723s (election complete) I0507 20:07:15.528329 1047 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:15.750721 1047 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:15.750829 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.299s (starting election) I0507 20:07:15.750936 1047 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:15.751376 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:15.751566 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:15.752405 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:15.826966 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.385s (SENT) I0507 20:07:15.827091 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:15.827448 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.437s (election complete) I0507 20:07:15.827555 1047 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:15.963389 407 maintenance_manager.cc:304] P d1bf0f72a88f4483a1906bc8246849dd: Scheduling FlushMRSOp(a0376a5e3b9f495aa00d366b35594a27): perf score=0.033384 I0507 20:07:15.964857 301 tablet.cc:1152] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering stage 1 (old memrowset already frozen for inserts) I0507 20:07:15.964942 301 compaction.cc:913] Selected 1 rowsets to compact: I0507 20:07:15.964983 301 compaction.cc:916] memrowset(current size on disk: ~0 bytes) I0507 20:07:15.965009 301 tablet.cc:1154] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Memstore in-memory size: 763 bytes I0507 20:07:15.965814 301 tablet.cc:1443] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6249363610975170560 or (T in {6249363610975170560})}] I0507 20:07:15.973246 301 multi_column_writer.cc:98] Opened CFile writers for 2 column(s) I0507 20:07:15.984108 301 tablet.cc:1528] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I0507 20:07:15.985337 301 tablet.cc:1583] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Flush Phase 2: carrying over any updates which arrived during Phase 1 I0507 20:07:15.985407 301 tablet.cc:1585] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6249363610975170560 or (T in {6249363610975170560})}] I0507 20:07:15.990154 301 tablet.cc:1633] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Flush successful on 6 rows (4492 bytes) I0507 20:07:15.990772 301 maintenance_manager.cc:494] P d1bf0f72a88f4483a1906bc8246849dd: FlushMRSOp(a0376a5e3b9f495aa00d366b35594a27) complete. Timing: real 0.027s user 0.024s sys 0.000s Metrics: {"cfile_init":1,"compiler_manager_pool.queue_time_us":385,"data dirs.queue_time_us":343,"data dirs.run_cpu_time_us":228,"data dirs.run_wall_time_us":1163,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4,"lbm_write_time_us":352,"lbm_writes_lt_1ms":18,"thread_start_us":624,"threads_started":3,"wal-append.queue_time_us":392} I0507 20:07:15.992372 407 maintenance_manager.cc:304] P d1bf0f72a88f4483a1906bc8246849dd: Scheduling UndoDeltaBlockGCOp(a0376a5e3b9f495aa00d366b35594a27): 223 bytes on disk I0507 20:07:15.993263 301 maintenance_manager.cc:494] P d1bf0f72a88f4483a1906bc8246849dd: UndoDeltaBlockGCOp(a0376a5e3b9f495aa00d366b35594a27) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":85,"lbm_reads_lt_1ms":4} I0507 20:07:15.993872 407 maintenance_manager.cc:304] P d1bf0f72a88f4483a1906bc8246849dd: Scheduling FlushMRSOp(7161aaf1a71d455883ef478337dfaf19): perf score=0.033376 I0507 20:07:15.994398 301 tablet.cc:1152] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering stage 1 (old memrowset already frozen for inserts) I0507 20:07:15.994468 301 compaction.cc:913] Selected 1 rowsets to compact: I0507 20:07:15.994505 301 compaction.cc:916] memrowset(current size on disk: ~0 bytes) I0507 20:07:15.994532 301 tablet.cc:1154] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Memstore in-memory size: 763 bytes I0507 20:07:15.994602 301 tablet.cc:1443] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6249363615459508224 or (T in {6249363615459508224})}] I0507 20:07:15.995577 301 multi_column_writer.cc:98] Opened CFile writers for 2 column(s) I0507 20:07:16.000417 301 tablet.cc:1528] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I0507 20:07:16.000655 301 tablet.cc:1583] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Flush Phase 2: carrying over any updates which arrived during Phase 1 I0507 20:07:16.000716 301 tablet.cc:1585] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6249363615459508224 or (T in {6249363615459508224})}] I0507 20:07:16.005028 301 tablet.cc:1633] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Flush successful on 10 rows (4506 bytes) I0507 20:07:16.005264 301 maintenance_manager.cc:494] P d1bf0f72a88f4483a1906bc8246849dd: FlushMRSOp(7161aaf1a71d455883ef478337dfaf19) complete. Timing: real 0.011s user 0.008s sys 0.000s Metrics: {"cfile_init":1,"data dirs.queue_time_us":68,"data dirs.run_cpu_time_us":264,"data dirs.run_wall_time_us":1944,"lbm_read_time_us":120,"lbm_reads_lt_1ms":4,"lbm_write_time_us":388,"lbm_writes_lt_1ms":18,"thread_start_us":290,"threads_started":1,"wal-append.queue_time_us":510} I0507 20:07:16.006081 407 maintenance_manager.cc:304] P d1bf0f72a88f4483a1906bc8246849dd: Scheduling FlushMRSOp(d80f4fae4b9f4d238f7f626507290390): perf score=0.033376 I0507 20:07:16.006538 301 tablet.cc:1152] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering stage 1 (old memrowset already frozen for inserts) I0507 20:07:16.006677 301 compaction.cc:913] Selected 1 rowsets to compact: I0507 20:07:16.006712 301 compaction.cc:916] memrowset(current size on disk: ~0 bytes) I0507 20:07:16.006772 301 tablet.cc:1154] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Memstore in-memory size: 763 bytes I0507 20:07:16.006817 301 tablet.cc:1443] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6249363615674081280 or (T in {6249363615674081280})}] I0507 20:07:16.008605 301 multi_column_writer.cc:98] Opened CFile writers for 2 column(s) I0507 20:07:16.011587 301 tablet.cc:1528] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I0507 20:07:16.011857 301 tablet.cc:1583] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Flush Phase 2: carrying over any updates which arrived during Phase 1 I0507 20:07:16.011898 301 tablet.cc:1585] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6249363615674081280 or (T in {6249363615674081280})}] I0507 20:07:16.015481 301 tablet.cc:1633] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Flush successful on 8 rows (4479 bytes) I0507 20:07:16.015719 301 maintenance_manager.cc:494] P d1bf0f72a88f4483a1906bc8246849dd: FlushMRSOp(d80f4fae4b9f4d238f7f626507290390) complete. Timing: real 0.009s user 0.008s sys 0.000s Metrics: {"cfile_init":1,"data dirs.queue_time_us":84,"data dirs.run_cpu_time_us":233,"data dirs.run_wall_time_us":1683,"lbm_read_time_us":54,"lbm_reads_lt_1ms":4,"lbm_write_time_us":350,"lbm_writes_lt_1ms":18,"thread_start_us":191,"threads_started":1,"wal-append.queue_time_us":359} I0507 20:07:16.016464 407 maintenance_manager.cc:304] P d1bf0f72a88f4483a1906bc8246849dd: Scheduling UndoDeltaBlockGCOp(7161aaf1a71d455883ef478337dfaf19): 274 bytes on disk I0507 20:07:16.016963 301 maintenance_manager.cc:494] P d1bf0f72a88f4483a1906bc8246849dd: UndoDeltaBlockGCOp(7161aaf1a71d455883ef478337dfaf19) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":67,"lbm_reads_lt_1ms":4} I0507 20:07:16.017328 407 maintenance_manager.cc:304] P d1bf0f72a88f4483a1906bc8246849dd: Scheduling UndoDeltaBlockGCOp(d80f4fae4b9f4d238f7f626507290390): 246 bytes on disk I0507 20:07:16.017726 301 maintenance_manager.cc:494] P d1bf0f72a88f4483a1906bc8246849dd: UndoDeltaBlockGCOp(d80f4fae4b9f4d238f7f626507290390) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":48,"lbm_reads_lt_1ms":4} I0507 20:07:16.050274 1047 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:16.050385 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.502s (starting election) I0507 20:07:16.050467 1047 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:16.050922 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:16.051097 1047 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:16.051328 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.299s (SENT) I0507 20:07:16.051376 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:16.051581 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.887s (election complete) I0507 20:07:16.051625 1047 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:16.052034 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:16.252305 1047 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:16.252418 1047 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.937s (starting election) I0507 20:07:16.252509 1047 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:16.252996 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:16.253212 1047 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:16.254601 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:16.265693 1047 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:16.265802 1047 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.795s (starting election) I0507 20:07:16.265933 1047 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:16.266412 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:16.266599 1047 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:16.267349 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:16.553958 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.502s (SENT) I0507 20:07:16.554047 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:16.554311 1047 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.614s (election complete) I0507 20:07:16.554388 1047 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:16.815560 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:13.816005 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:13.816269 (+ 264us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:13.816284 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:13.816380 (+ 96us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:16.815223 (+2998843us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":83} W0507 20:07:16.815827 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:17.062131 1060 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:17.062232 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.610s (starting election) I0507 20:07:17.062305 1060 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:17.062728 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:17.062906 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:17.062981 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.795s (SENT) I0507 20:07:17.063060 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:17.063205 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.480s (election complete) I0507 20:07:17.063256 1060 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:17.063668 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:17.169379 1060 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:17.169513 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.745s (starting election) I0507 20:07:17.169592 1060 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:17.170029 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:17.170193 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:17.171034 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:17.190804 1060 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:17.190924 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.832s (starting election) I0507 20:07:17.191007 1060 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:17.191408 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:17.191619 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:17.191869 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.937s (SENT) I0507 20:07:17.191922 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:17.192095 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.560s (election complete) I0507 20:07:17.192142 1060 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:17.192479 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:17.544284 1060 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:17.544401 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.510s (starting election) I0507 20:07:17.544489 1060 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:17.544955 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:17.545111 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:17.545852 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:17.674353 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.610s (SENT) I0507 20:07:17.674448 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:17.674695 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.499s (election complete) I0507 20:07:17.674764 1060 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:17.753121 1060 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:17.753244 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.418s (starting election) I0507 20:07:17.753334 1060 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:17.753859 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:17.754005 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:17.754647 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:17.915259 1060 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:17.915380 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.184s (starting election) I0507 20:07:17.915452 1060 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:17.915915 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:17.916069 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:17.916278 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.745s (SENT) I0507 20:07:17.916337 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:17.916539 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.816s (election complete) I0507 20:07:17.916590 1060 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:17.916864 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:18.024655 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.832s (SENT) I0507 20:07:18.024752 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:18.025023 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.400s (election complete) I0507 20:07:18.025092 1060 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:18.056552 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.510s (SENT) I0507 20:07:18.056637 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:18.056882 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.797s (election complete) I0507 20:07:18.056951 1060 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:18.101019 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.184s (SENT) I0507 20:07:18.101131 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:18.101500 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.523s (election complete) I0507 20:07:18.101575 1060 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:18.172857 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.418s (SENT) I0507 20:07:18.172945 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:18.173221 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.082s (election complete) I0507 20:07:18.173290 1060 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:18.256595 1060 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:18.256700 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.807s (starting election) I0507 20:07:18.256816 1060 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:18.257227 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:18.257398 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:18.258098 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:18.625802 1060 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:18.625916 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.595s (starting election) I0507 20:07:18.625993 1060 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:18.626451 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:18.626605 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:18.627421 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:18.855108 1060 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:18.855208 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.519s (starting election) I0507 20:07:18.855283 1060 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:18.855728 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:18.855878 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:18.856614 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:19.065043 1060 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:19.065150 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.116s (starting election) I0507 20:07:19.065232 1060 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:19.065702 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:19.065835 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.807s (SENT) I0507 20:07:19.065878 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:19.065892 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:19.066099 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.083s (election complete) I0507 20:07:19.066146 1060 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:19.066579 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:19.149987 1060 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:19.150107 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.641s (starting election) I0507 20:07:19.150192 1060 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:19.150691 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:19.150863 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:19.151582 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:19.182986 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.116s (SENT) I0507 20:07:19.183086 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:19.183346 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.616s (election complete) I0507 20:07:19.183413 1060 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:19.221643 1060 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:19.221757 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.731s (starting election) I0507 20:07:19.221855 1060 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:19.222286 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:19.222470 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:19.222712 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.595s (SENT) I0507 20:07:19.222777 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:19.223000 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.234s (election complete) I0507 20:07:19.223039 1060 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:19.223233 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:19.374490 1060 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:19.374577 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.233s (starting election) I0507 20:07:19.374650 1060 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:19.375074 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:19.375185 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:19.375851 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:19.376644 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.519s (SENT) I0507 20:07:19.376688 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:19.376839 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.651s (election complete) I0507 20:07:19.376883 1060 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:19.458073 1060 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:19.458169 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.523s (starting election) I0507 20:07:19.458233 1060 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:19.458689 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:19.458827 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:19.459523 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:19.608727 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.233s (SENT) I0507 20:07:19.608819 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:19.609164 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.497s (election complete) I0507 20:07:19.609333 1060 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:19.792780 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.641s (SENT) I0507 20:07:19.792873 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:19.793193 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.990s (election complete) I0507 20:07:19.793326 1060 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:19.955132 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.731s (SENT) I0507 20:07:19.955245 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:19.955611 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.202s (election complete) I0507 20:07:19.955756 1060 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:19.983641 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.523s (SENT) I0507 20:07:19.983770 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:19.984179 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.634s (election complete) I0507 20:07:19.984328 1060 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:20.107091 1060 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:20.107206 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.198s (starting election) I0507 20:07:20.107311 1060 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:20.107484 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:17.108173 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:17.108395 (+ 222us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:17.108412 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:17.108510 (+ 98us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:20.107202 (+2998692us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":75} W0507 20:07:20.107738 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:20.107754 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:20.107930 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:20.109283 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:20.305975 1060 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:20.306089 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.186s (starting election) I0507 20:07:20.306172 1060 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:20.306694 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:20.306824 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.198s (SENT) I0507 20:07:20.306851 1060 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:20.306993 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:20.307158 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.309s (election complete) I0507 20:07:20.307199 1060 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:20.307663 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:20.494204 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.186s (SENT) I0507 20:07:20.494309 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:20.494599 1060 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.796s (election complete) I0507 20:07:20.494686 1060 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:20.619328 1060 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:20.619446 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.405s (starting election) I0507 20:07:20.619523 1060 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:20.619985 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:20.620151 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:20.620944 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:20.783957 1060 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:20.784065 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.362s (starting election) I0507 20:07:20.784137 1060 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:20.784567 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:20.784725 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:20.785650 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:21.025770 1060 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:21.025821 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.405s (SENT) I0507 20:07:21.025866 1060 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.749s (starting election) I0507 20:07:21.025882 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:21.025939 1060 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:21.026377 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:21.026384 1066 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.287s (election complete) I0507 20:07:21.026424 1066 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:21.026520 1060 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:21.027140 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:21.147083 1060 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:21.147267 1060 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.133s (starting election) I0507 20:07:21.147352 1060 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:21.147730 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.362s (SENT) I0507 20:07:21.147827 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:21.147895 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:21.148082 1060 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:21.148198 1066 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.952s (election complete) I0507 20:07:21.148258 1066 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:21.148856 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:21.281941 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.133s (SENT) I0507 20:07:21.282038 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:21.282352 1066 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.803s (election complete) I0507 20:07:21.282425 1066 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:21.291465 1066 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:21.291571 1066 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.569s (starting election) I0507 20:07:21.291651 1066 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:21.292117 1066 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:21.292300 1066 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:21.293041 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:21.314543 1066 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:21.314652 1066 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.497s (starting election) I0507 20:07:21.314725 1066 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:21.315224 1066 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:21.315384 1066 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:21.316114 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:21.776770 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.749s (SENT) I0507 20:07:21.776916 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:21.777222 1066 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.248s (election complete) I0507 20:07:21.777292 1066 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:21.813117 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.497s (SENT) I0507 20:07:21.813215 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:21.813503 1066 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.643s (election complete) I0507 20:07:21.813573 1066 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:21.861583 1066 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:21.861685 1066 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.680s (starting election) I0507 20:07:21.861753 1066 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:21.862069 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.569s (SENT) I0507 20:07:21.862146 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:21.862192 1066 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:21.862337 1066 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:21.862469 1066 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.609s (election complete) I0507 20:07:21.862517 1066 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:21.863381 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:22.086251 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:22.086365 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.840s (starting election) I0507 20:07:22.086441 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:22.086901 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:22.087056 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:22.087777 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:22.457695 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:22.457834 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.685s (starting election) I0507 20:07:22.457932 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:22.458361 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:22.458537 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:22.459306 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:22.472203 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:22.472314 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.637s (starting election) I0507 20:07:22.472441 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:22.472874 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:22.473047 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:22.473765 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:22.544162 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.680s (SENT) I0507 20:07:22.544261 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:22.544550 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.238s (election complete) I0507 20:07:22.544646 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:22.783211 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:22.783339 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.856s (starting election) I0507 20:07:22.783447 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:22.783952 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:22.784127 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:22.784797 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:22.927403 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:22.927556 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.538s (starting election) W0507 20:07:22.927654 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.840s (SENT) I0507 20:07:22.927744 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:22.927670 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:22.928264 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:22.928263 1069 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.536s (election complete) I0507 20:07:22.928349 1069 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:22.928490 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:22.929133 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:23.111032 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.637s (SENT) I0507 20:07:23.111155 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:23.111497 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.298s (election complete) I0507 20:07:23.111613 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:23.144089 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:23.144239 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.924s (starting election) I0507 20:07:23.144368 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:23.144973 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:23.145150 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.685s (SENT) I0507 20:07:23.145210 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:23.145447 1069 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.632s (election complete) I0507 20:07:23.145510 1069 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:23.155275 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:23.156144 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:23.410732 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:23.410850 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.930s (starting election) I0507 20:07:23.410938 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:23.411327 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:23.411455 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:23.412140 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:23.415419 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:20.417528 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:20.417887 (+ 359us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:20.417905 (+ 18us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:20.418022 (+ 117us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:23.415153 (+2997131us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":94} W0507 20:07:23.415678 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:23.464889 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:23.464984 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (starting election) I0507 20:07:23.465067 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:23.465451 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:23.465571 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:23.466223 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:23.467000 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.538s (SENT) I0507 20:07:23.467049 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:23.467214 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.302s (election complete) I0507 20:07:23.467267 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:23.565008 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.099s (SENT) I0507 20:07:23.565121 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:23.565450 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.578s (election complete) I0507 20:07:23.565515 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:23.642233 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.856s (SENT) I0507 20:07:23.642338 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:23.642660 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.947s (election complete) I0507 20:07:23.642738 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:23.778687 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:23.778795 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.192s (starting election) I0507 20:07:23.778872 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:23.779346 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:23.779505 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:23.780303 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:23.971709 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:23.971740 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.192s (SENT) I0507 20:07:23.971822 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.220s (starting election) I0507 20:07:23.971909 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:23.971988 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:23.972378 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:23.972447 1072 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.484s (election complete) I0507 20:07:23.972496 1072 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:23.972519 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:23.973260 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:24.080651 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.924s (SENT) I0507 20:07:24.080740 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:24.081009 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.782s (election complete) I0507 20:07:24.081076 1067 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:24.144697 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:24.144819 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.578s (starting election) I0507 20:07:24.144906 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:24.145417 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:24.145618 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:24.146322 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:24.192947 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.220s (SENT) I0507 20:07:24.193053 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:24.193361 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.876s (election complete) I0507 20:07:24.193439 1067 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:24.343194 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.930s (SENT) I0507 20:07:24.343305 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:24.343570 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.610s (election complete) I0507 20:07:24.343642 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:24.724164 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:24.724274 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.265s (starting election) I0507 20:07:24.724376 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:24.724807 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:24.724988 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:24.725261 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.578s (SENT) I0507 20:07:24.725306 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:24.725476 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.369s (election complete) I0507 20:07:24.725510 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:24.725960 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:24.954741 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:24.954840 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.690s (starting election) I0507 20:07:24.954953 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:24.955410 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:24.955576 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:24.956254 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:24.990312 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.265s (SENT) I0507 20:07:24.990417 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:24.990698 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.524s (election complete) I0507 20:07:24.990772 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:25.069744 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:25.069844 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.311s (starting election) I0507 20:07:25.069907 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:25.070312 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:25.070472 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:25.071198 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:25.381937 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:25.381985 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.311s (SENT) I0507 20:07:25.382052 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.224s (starting election) I0507 20:07:25.382079 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:25.382119 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:25.382553 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:25.382558 1074 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.884s (election complete) I0507 20:07:25.382612 1074 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:25.382704 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:25.383352 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:25.516098 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:25.516206 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.240s (starting election) I0507 20:07:25.516280 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:25.516685 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:25.516835 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:25.517573 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:25.606941 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.224s (SENT) I0507 20:07:25.607028 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:25.607270 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.994s (election complete) I0507 20:07:25.607331 1067 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:25.646273 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:25.646410 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.479s (starting election) I0507 20:07:25.646503 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:25.647013 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:25.647197 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:25.647255 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.690s (SENT) I0507 20:07:25.647332 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:25.647491 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.766s (election complete) I0507 20:07:25.647541 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:25.647977 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:25.757117 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:25.757239 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.786s (starting election) I0507 20:07:25.757321 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:25.757448 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.240s (SENT) I0507 20:07:25.757521 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:25.757737 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:25.757746 1074 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.158s (election complete) I0507 20:07:25.757789 1074 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:25.757889 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:25.758477 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:25.916400 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:25.916498 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.726s (starting election) I0507 20:07:25.916571 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:25.916959 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:25.917102 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:25.917738 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:26.127198 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.479s (SENT) I0507 20:07:26.127296 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:26.127547 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.253s (election complete) I0507 20:07:26.127604 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:26.381012 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:26.381116 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.758s (starting election) I0507 20:07:26.381187 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:26.381614 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:26.381754 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:26.382410 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:26.546052 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.786s (SENT) I0507 20:07:26.546144 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:26.546386 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.472s (election complete) I0507 20:07:26.546452 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:26.602699 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:26.602802 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.218s (starting election) I0507 20:07:26.602897 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:26.603286 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:26.603436 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:26.604095 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:26.644192 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.726s (SENT) I0507 20:07:26.644281 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:26.644546 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.118s (election complete) I0507 20:07:26.644629 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:26.735427 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:23.734711 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:23.734946 (+ 235us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:23.734959 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:23.735073 (+ 114us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:26.735152 (+3000079us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":68} W0507 20:07:26.735688 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:26.763519 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:26.763629 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.575s (starting election) I0507 20:07:26.763700 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:26.764123 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:26.764272 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:26.765067 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:26.821357 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:26.821465 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.414s (starting election) I0507 20:07:26.821540 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:26.821980 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:26.822142 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:26.822363 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.218s (SENT) I0507 20:07:26.822422 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:26.822599 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.841s (election complete) I0507 20:07:26.822649 1067 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:26.823072 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:27.140105 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:27.140197 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.415s (starting election) I0507 20:07:27.140265 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:27.140657 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:27.140785 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:27.140981 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.758s (SENT) I0507 20:07:27.141026 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:27.141219 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.377s (election complete) I0507 20:07:27.141273 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:27.141384 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:27.237187 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.414s (SENT) I0507 20:07:27.237288 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:27.237562 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.747s (election complete) I0507 20:07:27.237640 1067 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:27.339500 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:27.339610 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.717s (starting election) I0507 20:07:27.339709 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:27.340210 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:27.340381 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:27.340647 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.575s (SENT) I0507 20:07:27.340694 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:27.340872 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.521s (election complete) I0507 20:07:27.340934 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:27.341110 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:27.519634 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:27.519773 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.370s (starting election) I0507 20:07:27.519891 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:27.520414 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:27.520622 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:27.521363 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:27.557037 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.415s (SENT) I0507 20:07:27.557127 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:27.557382 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.651s (election complete) I0507 20:07:27.557453 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:27.862802 1067 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:27.862937 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (starting election) I0507 20:07:27.863018 1067 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:27.863464 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:27.863636 1067 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:27.864490 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:27.891697 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.370s (SENT) I0507 20:07:27.891805 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:27.892073 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.593s (election complete) I0507 20:07:27.892143 1067 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:27.985671 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:27.985786 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.701s (starting election) I0507 20:07:27.985862 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:27.986302 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:27.986467 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:27.987275 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:28.058799 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.717s (SENT) I0507 20:07:28.058921 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:28.059214 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.948s (election complete) I0507 20:07:28.059285 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:28.485821 1067 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:28.485935 1067 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.838s (starting election) I0507 20:07:28.486018 1067 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:28.486526 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:28.486737 1067 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:28.487674 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:28.687180 1067 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:28.687301 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.714s (starting election) I0507 20:07:28.687381 1067 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:28.687846 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:28.688040 1067 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:28.688874 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:28.689469 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.701s (SENT) I0507 20:07:28.689518 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:28.689693 1067 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.923s (election complete) I0507 20:07:28.689749 1067 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:28.727371 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.863s (SENT) I0507 20:07:28.727470 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:28.727821 1067 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.967s (election complete) I0507 20:07:28.727907 1067 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:29.325170 1077 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:29.325258 1077 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.675s (starting election) I0507 20:07:29.325330 1077 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:29.325515 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.838s (SENT) I0507 20:07:29.325589 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:29.325749 1077 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:29.326000 1078 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.444s (election complete) I0507 20:07:29.326076 1078 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:29.326498 1077 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:29.326771 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:29.403220 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.714s (SENT) I0507 20:07:29.403309 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:29.403604 1077 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.111s (election complete) I0507 20:07:29.403678 1077 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:29.515735 1077 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:29.515851 1077 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.400s (starting election) I0507 20:07:29.515926 1077 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:29.516378 1077 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:29.516582 1077 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:29.517591 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:29.696317 1077 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:29.696475 1077 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.521s (starting election) I0507 20:07:29.696564 1077 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:29.697003 1077 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:29.697193 1077 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:29.698101 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:29.771500 1077 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:29.771639 1077 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.130s (starting election) I0507 20:07:29.771746 1077 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:29.772323 1077 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:29.772537 1077 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:29.773663 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:29.902870 1077 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:29.902992 1077 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.864s (starting election) I0507 20:07:29.903075 1077 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:29.903507 1077 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:29.903681 1077 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:29.903821 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.130s (SENT) I0507 20:07:29.903928 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:07:29.904589 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:29.904616 1077 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.733s (election complete) I0507 20:07:29.904686 1077 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:29.916811 1077 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:29.916882 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.400s (SENT) I0507 20:07:29.916946 1077 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.538s (starting election) I0507 20:07:29.916996 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:29.917049 1077 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:29.917513 1077 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:29.917724 1077 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:29.917876 1077 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.661s (election complete) I0507 20:07:29.917946 1077 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:29.918787 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:30.002627 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.675s (SENT) I0507 20:07:30.002740 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:30.003173 1080 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.616s (election complete) I0507 20:07:30.003279 1080 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:30.059586 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:27.059638 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:27.059915 (+ 277us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:27.059932 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:27.060057 (+ 125us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:30.059305 (+2999248us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":92} W0507 20:07:30.059939 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:07:30.218974 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.521s (SENT) I0507 20:07:30.218974 1080 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:30.219135 1080 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.181s (starting election) I0507 20:07:30.219225 1080 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:30.219689 1080 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:30.219862 1080 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:30.219075 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:30.220161 1080 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.112s (election complete) I0507 20:07:30.220219 1080 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:30.220666 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:30.333289 1080 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:30.333441 1080 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.543s (starting election) I0507 20:07:30.333537 1080 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:30.334115 1080 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:30.334364 1080 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:30.335139 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:30.402196 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.181s (SENT) I0507 20:07:30.402287 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:30.402585 1080 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.109s (election complete) I0507 20:07:30.402657 1080 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:30.457906 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.538s (SENT) I0507 20:07:30.457999 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:30.458278 1080 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.083s (election complete) I0507 20:07:30.458370 1080 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:30.512722 1080 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:30.512842 1080 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.233s (starting election) I0507 20:07:30.512941 1080 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:30.513525 1080 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:30.513710 1080 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:30.514452 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:30.542583 1080 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:30.542696 1080 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.971s (starting election) I0507 20:07:30.542776 1080 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:30.543246 1080 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:30.543429 1080 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:30.544327 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:30.620753 1080 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:30.620853 1080 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.505s (starting election) I0507 20:07:30.620934 1080 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:30.621384 1080 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:30.621543 1080 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:30.622174 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:30.746824 1080 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:30.746948 1080 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.831s (starting election) W0507 20:07:30.746978 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.233s (SENT) I0507 20:07:30.747040 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:30.747017 1080 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:30.747406 1080 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:30.747575 1080 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:30.747656 1083 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.424s (election complete) I0507 20:07:30.747782 1083 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:30.748265 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:30.770386 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.864s (SENT) I0507 20:07:30.770490 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:30.770849 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.079s (election complete) I0507 20:07:30.770983 1083 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:30.850417 1083 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:30.850522 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.120s (starting election) I0507 20:07:30.850597 1083 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:30.851114 1083 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:30.851284 1083 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:30.852088 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:30.878233 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.543s (SENT) I0507 20:07:30.878334 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:30.878674 1083 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.642s (election complete) I0507 20:07:30.878747 1083 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:30.971117 1083 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:30.971230 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.772s (starting election) I0507 20:07:30.971313 1083 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:30.971798 1083 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:30.972002 1083 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:30.972337 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.120s (SENT) I0507 20:07:30.972398 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:30.972800 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.817s (election complete) W0507 20:07:30.972816 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:30.972868 1083 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:31.128254 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.505s (SENT) I0507 20:07:31.128348 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:31.128661 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.566s (election complete) I0507 20:07:31.128733 1083 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:31.514441 1083 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:31.514569 1083 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.552s (starting election) I0507 20:07:31.514703 1083 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:31.515246 1083 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:31.515455 1083 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:31.515530 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.971s (SENT) I0507 20:07:31.515589 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:31.515738 1083 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.653s (election complete) I0507 20:07:31.515771 1083 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:31.516863 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:31.521415 1083 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:31.521492 1083 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.381s (starting election) I0507 20:07:31.521574 1083 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:31.521996 1083 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:31.522128 1083 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:31.522961 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:31.579691 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.831s (SENT) I0507 20:07:31.579795 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:31.580104 1083 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.732s (election complete) I0507 20:07:31.580200 1083 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:31.695806 1083 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:31.695941 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.494s (starting election) I0507 20:07:31.696063 1083 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:31.696707 1083 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:31.696907 1083 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:31.697644 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:31.745415 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.772s (SENT) I0507 20:07:31.745523 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:31.745854 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.950s (election complete) I0507 20:07:31.745949 1083 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:31.903728 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.381s (SENT) I0507 20:07:31.903841 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:31.904234 1083 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.606s (election complete) I0507 20:07:31.904332 1083 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:32.068792 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.552s (SENT) I0507 20:07:32.068904 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:32.072088 1083 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.447s (election complete) I0507 20:07:32.072172 1083 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:32.191231 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.494s (SENT) I0507 20:07:32.191354 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:32.191612 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.194s (election complete) I0507 20:07:32.191682 1083 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:32.386590 1083 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:32.386693 1083 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.536s (starting election) I0507 20:07:32.386766 1083 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:32.387185 1083 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:32.387336 1083 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:32.389154 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:32.511209 1083 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:32.511317 1083 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.386s (starting election) I0507 20:07:32.511396 1083 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:32.511785 1083 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:32.511952 1083 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:32.512881 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:32.519701 1083 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:32.519820 1083 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.585s (starting election) I0507 20:07:32.519889 1083 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:32.520280 1083 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:32.520431 1083 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:32.521270 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:32.898244 1083 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:32.898352 1083 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.384s (starting election) I0507 20:07:32.898428 1083 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:32.898682 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.386s (SENT) I0507 20:07:32.898741 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:32.898902 1083 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:32.899062 1083 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:32.899193 1086 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.952s (election complete) I0507 20:07:32.899237 1086 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:32.899643 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:32.923125 1086 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:32.923229 1086 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.950s (starting election) I0507 20:07:32.923308 1086 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:32.923815 1086 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:32.923983 1086 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:32.924273 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.536s (SENT) I0507 20:07:32.924351 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:32.924747 1086 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.420s (election complete) I0507 20:07:32.924816 1086 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:32.924806 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:33.105981 1086 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:33.106106 1086 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.956s (starting election) W0507 20:07:33.106146 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.585s (SENT) I0507 20:07:33.106240 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:33.106182 1086 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:33.106665 1086 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:33.106675 1083 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.638s (election complete) I0507 20:07:33.106782 1083 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:33.106895 1086 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:33.107640 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:33.284000 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.384s (SENT) I0507 20:07:33.284103 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:33.284462 1086 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.688s (election complete) I0507 20:07:33.284546 1086 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:33.345795 1086 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:33.345917 1086 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.289s (starting election) I0507 20:07:33.346006 1086 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:33.346468 1086 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:33.346632 1086 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:33.347590 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:33.395496 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:30.396559 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:30.396858 (+ 299us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:30.396875 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:30.396984 (+ 109us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:33.395173 (+2998189us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":78} W0507 20:07:33.395781 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:33.636315 1086 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:33.636485 1086 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.568s (starting election) I0507 20:07:33.636590 1086 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:33.636780 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.289s (SENT) I0507 20:07:33.636912 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:33.637121 1086 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:33.637495 1086 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:33.637630 1088 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.688s (election complete) I0507 20:07:33.637689 1088 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:33.638206 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:33.745546 1088 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:33.745678 1088 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.746s (starting election) I0507 20:07:33.745807 1088 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:33.746299 1088 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:33.746491 1088 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:33.747345 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:33.875442 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.950s (SENT) I0507 20:07:33.875571 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:33.875927 1088 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.866s (election complete) I0507 20:07:33.876021 1088 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:33.973831 1088 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:33.973948 1088 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.190s (starting election) I0507 20:07:33.974050 1088 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:33.974592 1088 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:33.974813 1088 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:33.975617 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:34.064384 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.956s (SENT) I0507 20:07:34.064491 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:34.064824 1088 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.717s (election complete) I0507 20:07:34.064941 1088 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:34.164921 1088 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:34.165014 1088 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.825s (starting election) I0507 20:07:34.165081 1088 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:34.165496 1088 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:34.165578 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.190s (SENT) I0507 20:07:34.165634 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:34.165647 1088 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:34.165796 1088 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.780s (election complete) I0507 20:07:34.165827 1088 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:34.166293 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:34.207433 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.568s (SENT) I0507 20:07:34.207531 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:34.207823 1088 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.161s (election complete) I0507 20:07:34.207895 1088 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:34.369935 1088 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:34.370044 1088 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.494s (starting election) I0507 20:07:34.370172 1088 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:34.370685 1088 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:34.370857 1088 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:34.371538 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:34.494089 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.746s (SENT) I0507 20:07:34.494189 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:34.494482 1088 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.151s (election complete) I0507 20:07:34.494572 1088 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:34.645818 1088 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:34.645920 1088 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.866s (starting election) I0507 20:07:34.646001 1088 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:34.646486 1088 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:34.646608 1088 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:34.647378 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:34.864964 1088 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:34.865090 1088 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.508s (starting election) I0507 20:07:34.865186 1088 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:34.865612 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.494s (SENT) I0507 20:07:34.865700 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:34.865731 1088 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:34.865887 1088 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:34.865993 1088 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.812s (election complete) I0507 20:07:34.866029 1088 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:34.866549 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:34.946458 1090 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:34.946596 1090 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.515s (starting election) I0507 20:07:34.946674 1090 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:34.947106 1090 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:34.947295 1090 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:34.947993 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:34.991875 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.825s (SENT) I0507 20:07:34.991971 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:34.992274 1090 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.953s (election complete) I0507 20:07:34.992352 1090 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:35.375298 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.508s (SENT) I0507 20:07:35.375385 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:35.375679 1090 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.538s (election complete) I0507 20:07:35.375751 1090 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:35.463326 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.515s (SENT) I0507 20:07:35.463414 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:35.463692 1090 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.450s (election complete) I0507 20:07:35.463753 1090 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:35.513106 1090 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:35.513221 1090 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.201s (starting election) I0507 20:07:35.513299 1090 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:35.513700 1090 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:35.513854 1090 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:35.514045 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.866s (SENT) I0507 20:07:35.514103 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:35.514256 1090 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.475s (election complete) I0507 20:07:35.514293 1090 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:35.514806 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:35.715628 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.201s (SENT) I0507 20:07:35.715751 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:35.716035 1090 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.718s (election complete) I0507 20:07:35.716097 1090 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:35.914615 1090 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:35.914721 1090 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.534s (starting election) I0507 20:07:35.914801 1090 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:35.915180 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:35.915273 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.419s (starting election) I0507 20:07:35.915434 1090 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:35.915343 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:35.915606 1090 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:35.915771 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:35.915935 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:35.916357 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:35.916435 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:36.335047 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:36.335171 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.698s (starting election) I0507 20:07:36.335253 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:36.335598 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.419s (SENT) I0507 20:07:36.335662 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:36.335731 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:36.335840 1090 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.460s (election complete) I0507 20:07:36.335885 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:36.335883 1090 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:36.336541 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:36.434813 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:36.434926 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.984s (starting election) I0507 20:07:36.435017 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:36.435485 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:36.435624 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:36.436393 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:36.449390 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:36.449529 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.735s (starting election) I0507 20:07:36.449605 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:36.450021 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:36.450218 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:36.450238 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.534s (SENT) I0507 20:07:36.450330 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:36.450539 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.339s (election complete) I0507 20:07:36.450613 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:36.450968 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:36.735461 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:33.736713 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:33.736934 (+ 221us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:33.736946 (+ 12us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:33.737033 (+ 87us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:36.735212 (+2998179us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":72} W0507 20:07:36.735709 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:36.790129 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:36.790257 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.285s (starting election) I0507 20:07:36.790349 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:36.790999 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:36.791178 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:36.792054 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:36.796959 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:36.797055 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.742s (starting election) I0507 20:07:36.797154 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:36.797607 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:36.797761 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:36.798487 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:37.034834 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.698s (SENT) I0507 20:07:37.034957 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:37.035339 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.267s (election complete) I0507 20:07:37.035425 1092 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:37.075599 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:37.075716 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.680s (starting election) I0507 20:07:37.075794 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:37.076341 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:37.076503 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:37.076761 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.285s (SENT) I0507 20:07:37.076823 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:37.077026 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.437s (election complete) I0507 20:07:37.077096 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:37.077268 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:37.186326 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.735s (SENT) I0507 20:07:37.186514 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:37.186839 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.730s (election complete) I0507 20:07:37.186938 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:37.303601 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:37.303721 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.890s (starting election) I0507 20:07:37.303848 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:37.304296 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:37.304467 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:37.305265 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:37.419490 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:37.419618 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.606s (starting election) I0507 20:07:37.419723 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:37.420356 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:37.420563 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:37.420609 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.984s (SENT) I0507 20:07:37.420680 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:37.420914 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.756s (election complete) I0507 20:07:37.421000 1092 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:37.421476 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:37.540547 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.742s (SENT) I0507 20:07:37.540634 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:37.540988 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.589s (election complete) I0507 20:07:37.541119 1092 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:37.758146 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.680s (SENT) I0507 20:07:37.758255 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:37.758656 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.392s (election complete) I0507 20:07:37.758831 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:38.028281 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.606s (SENT) I0507 20:07:38.028381 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:38.028884 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.335s (election complete) I0507 20:07:38.029034 1092 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:38.131047 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:38.131219 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.711s (starting election) I0507 20:07:38.131359 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:38.132087 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:38.132339 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:38.133323 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:38.151374 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:38.151541 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.525s (starting election) I0507 20:07:38.151669 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:38.152333 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:38.152560 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:38.153697 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:38.195375 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.890s (SENT) I0507 20:07:38.195487 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:38.195919 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.925s (election complete) I0507 20:07:38.196071 1092 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:38.365242 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:38.365463 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.528s (starting election) I0507 20:07:38.365584 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:38.366291 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:38.366559 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:38.367790 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:38.677515 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:38.677640 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.772s (starting election) I0507 20:07:38.677726 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:38.678302 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:38.678463 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:38.678719 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.525s (SENT) I0507 20:07:38.678772 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:38.679038 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.969s (election complete) I0507 20:07:38.679184 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:38.679427 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:38.845146 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.711s (SENT) I0507 20:07:38.845245 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:38.845558 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.965s (election complete) I0507 20:07:38.845629 1092 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:38.894999 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:38.895076 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.528s (SENT) I0507 20:07:38.895105 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.092s (starting election) I0507 20:07:38.895153 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:38.895175 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:38.895628 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:38.895633 1096 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.317s (election complete) I0507 20:07:38.895838 1096 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:38.895938 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:38.897158 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:38.989099 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.092s (SENT) I0507 20:07:38.989199 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:38.989480 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.277s (election complete) I0507 20:07:38.989553 1092 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:39.267519 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:39.267658 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.840s (starting election) I0507 20:07:39.267765 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:39.268369 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:39.268609 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:39.269531 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:39.451197 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.772s (SENT) I0507 20:07:39.451304 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:39.451591 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.297s (election complete) I0507 20:07:39.451683 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:39.749030 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:39.749150 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.654s (starting election) I0507 20:07:39.749233 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:39.749775 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:39.749975 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:39.750718 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:39.811022 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:39.811147 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.974s (starting election) I0507 20:07:39.811245 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:39.811836 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:39.812041 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:39.812855 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:40.087517 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:37.088370 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:37.088641 (+ 271us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:37.088656 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:37.088770 (+ 114us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:40.087227 (+2998457us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":101} W0507 20:07:40.087827 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:40.108167 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:40.108315 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.861s (starting election) I0507 20:07:40.108417 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:40.108989 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:40.109220 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:40.109282 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.840s (SENT) I0507 20:07:40.109354 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:40.109599 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.390s (election complete) I0507 20:07:40.109666 1092 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:40.110170 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:40.403913 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:40.404031 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.205s (starting election) I0507 20:07:40.404116 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:40.404374 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.654s (SENT) I0507 20:07:40.404443 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:40.404644 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:40.404826 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:40.404966 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.914s (election complete) I0507 20:07:40.405016 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:40.405691 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:40.500458 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:40.500571 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.134s (starting election) I0507 20:07:40.500649 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:40.501171 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:40.501336 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:40.502224 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:40.610332 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.205s (SENT) I0507 20:07:40.610437 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:40.610724 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.506s (election complete) I0507 20:07:40.610796 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:40.635643 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:40.635705 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.134s (SENT) I0507 20:07:40.635742 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.079s (starting election) I0507 20:07:40.635769 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:40.635821 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:40.636265 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:40.636301 1098 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.156s (election complete) I0507 20:07:40.636394 1098 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:40.636430 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:40.637338 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:40.715903 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.079s (SENT) I0507 20:07:40.715997 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:40.716297 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.088s (election complete) I0507 20:07:40.716369 1092 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:40.785847 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:40.785959 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.800s (starting election) I0507 20:07:40.786039 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:40.786465 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:40.786615 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:40.786901 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.974s (SENT) I0507 20:07:40.786964 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:40.787096 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.212s (election complete) I0507 20:07:40.787132 1092 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:40.787359 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:40.805693 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:40.805795 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.826s (starting election) I0507 20:07:40.805868 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:40.806285 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:40.806432 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:40.807130 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:40.972390 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.861s (SENT) I0507 20:07:40.972492 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:40.972868 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.335s (election complete) I0507 20:07:40.972939 1092 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:40.999531 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:40.999725 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.916s (starting election) I0507 20:07:40.999816 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:41.000262 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:41.000413 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:41.001164 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:41.118060 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:41.118180 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.668s (starting election) I0507 20:07:41.118255 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:41.118741 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:41.118937 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:41.119715 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:41.309245 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:41.309362 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.475s (starting election) I0507 20:07:41.309440 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:41.309916 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:41.310086 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:41.310926 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:41.588769 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.800s (SENT) I0507 20:07:41.588863 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:41.589109 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.086s (election complete) I0507 20:07:41.589172 1092 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:41.633420 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.826s (SENT) I0507 20:07:41.633658 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:41.633960 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.829s (election complete) I0507 20:07:41.634048 1092 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:41.675787 1092 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:41.675906 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.781s (starting election) I0507 20:07:41.676007 1092 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:41.676551 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:41.676735 1092 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:41.677528 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:41.785603 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.475s (SENT) I0507 20:07:41.785684 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:41.785912 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.620s (election complete) I0507 20:07:41.785975 1092 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:41.786783 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:41.786821 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.652s (starting election) I0507 20:07:41.786870 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:41.787284 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:41.787405 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:41.787612 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.668s (SENT) I0507 20:07:41.787655 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:41.787808 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.328s (election complete) I0507 20:07:41.787869 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:41.788050 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:41.917356 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.916s (SENT) I0507 20:07:41.917450 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:41.917754 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.562s (election complete) I0507 20:07:41.917822 1092 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:42.116421 1092 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:42.116549 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.382s (starting election) I0507 20:07:42.116657 1092 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:42.117362 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:42.117576 1092 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:42.118399 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:42.406985 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:42.407095 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.370s (starting election) I0507 20:07:42.407178 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:42.407646 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:42.407819 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:42.408641 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:42.440820 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.652s (SENT) I0507 20:07:42.440927 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:42.441222 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.505s (election complete) I0507 20:07:42.441292 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:42.458956 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.781s (SENT) I0507 20:07:42.459058 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:42.459353 1092 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.389s (election complete) I0507 20:07:42.459447 1092 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:42.500210 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.382s (SENT) I0507 20:07:42.500329 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:42.500645 1092 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.618s (election complete) I0507 20:07:42.500707 1092 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:42.777837 1092 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:42.777926 1092 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.501s (starting election) W0507 20:07:42.777948 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.370s (SENT) I0507 20:07:42.778007 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:42.777984 1092 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:42.778374 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:42.778481 1092 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:42.778506 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.116s (election complete) I0507 20:07:42.778570 1100 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:42.779242 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:42.849071 1100 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:42.849186 1100 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.909s (starting election) I0507 20:07:42.849262 1100 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:42.849719 1100 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:42.849941 1100 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:42.850780 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:42.895769 1100 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:42.895917 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.475s (starting election) I0507 20:07:42.896019 1100 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:42.896437 1100 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:42.896611 1100 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:42.897363 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:43.119498 1100 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:43.119611 1100 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.476s (starting election) I0507 20:07:43.119729 1100 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:43.120226 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:43.120431 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:43.121186 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:43.279856 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.501s (SENT) I0507 20:07:43.279961 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:43.280212 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.444s (election complete) I0507 20:07:43.280293 1100 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:43.372191 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.475s (SENT) I0507 20:07:43.372299 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:43.372643 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.165s (election complete) I0507 20:07:43.372740 1100 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:43.443491 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:40.445891 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:40.446150 (+ 259us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:40.446163 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:40.446257 (+ 94us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:43.443170 (+2996913us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":90} W0507 20:07:43.443733 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:43.539104 1100 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:43.539230 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.820s (starting election) I0507 20:07:43.539320 1100 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:43.539830 1100 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:43.540011 1100 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:43.540899 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:43.596890 1100 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:43.596976 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.476s (SENT) I0507 20:07:43.597030 1100 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.707s (starting election) I0507 20:07:43.597065 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:43.597163 1100 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:43.597695 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:43.597700 1102 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.456s (election complete) I0507 20:07:43.597774 1102 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:43.597888 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:43.598749 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:43.759727 1100 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:43.759882 1100 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.767s (starting election) W0507 20:07:43.759976 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.909s (SENT) I0507 20:07:43.760085 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:43.759989 1100 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:43.760526 1100 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:43.760547 1102 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.256s (election complete) I0507 20:07:43.760625 1102 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:43.760725 1100 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:43.761502 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:44.018137 1100 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:44.018282 1100 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.545s (starting election) I0507 20:07:44.018402 1100 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:44.019028 1100 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:44.019265 1100 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:44.020218 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:44.055197 1100 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:44.055316 1100 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.366s (starting election) I0507 20:07:44.055411 1100 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:44.055974 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:44.056157 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:44.057096 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:44.305436 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.707s (SENT) I0507 20:07:44.305567 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:44.305874 1100 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.427s (election complete) I0507 20:07:44.305959 1100 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:44.360016 1100 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:44.360136 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.225s (starting election) I0507 20:07:44.360229 1100 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:44.360720 1100 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:44.360906 1100 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:44.361158 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.820s (SENT) I0507 20:07:44.361232 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:44.361434 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.606s (election complete) I0507 20:07:44.361505 1100 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:44.361717 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:44.423467 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.366s (SENT) I0507 20:07:44.423590 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:44.423944 1100 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.917s (election complete) I0507 20:07:44.424026 1100 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:44.529029 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.767s (SENT) I0507 20:07:44.529122 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:44.529412 1100 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.637s (election complete) I0507 20:07:44.529492 1100 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:44.564704 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.545s (SENT) I0507 20:07:44.564801 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:44.565168 1100 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.706s (election complete) I0507 20:07:44.565259 1100 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:44.586190 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.225s (SENT) I0507 20:07:44.586280 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:44.586576 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.476s (election complete) I0507 20:07:44.586668 1100 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:45.063797 1100 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:45.063913 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.685s (starting election) I0507 20:07:45.064002 1100 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:45.064422 1100 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:45.064586 1100 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:45.065570 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:45.272243 1100 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:45.272373 1100 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.223s (starting election) I0507 20:07:45.272464 1100 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:45.272943 1100 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:45.273142 1100 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:45.274160 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:45.341671 1100 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:45.341835 1100 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.081s (starting election) I0507 20:07:45.341964 1100 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:45.342533 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:45.342736 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:45.343858 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:45.423636 1100 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:45.423746 1100 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.661s (starting election) I0507 20:07:45.423840 1100 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:45.424343 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.081s (SENT) I0507 20:07:45.424362 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:45.424481 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:45.424638 1100 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:45.424787 1100 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.701s (election complete) I0507 20:07:45.424840 1100 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:45.425503 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:45.496206 1105 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:45.496331 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.561s (starting election) I0507 20:07:45.496425 1105 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:45.496671 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.223s (SENT) I0507 20:07:45.496767 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:45.496888 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:45.497061 1100 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.383s (election complete) I0507 20:07:45.497102 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:45.497138 1100 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:45.498769 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:45.750329 1105 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:45.750478 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.860s (starting election) W0507 20:07:45.750476 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.685s (SENT) I0507 20:07:45.750632 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:45.750600 1105 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:45.751085 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:45.751114 1100 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.719s (election complete) I0507 20:07:45.751188 1100 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:45.751294 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:45.752113 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:45.880564 1105 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:45.880791 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.409s (starting election) I0507 20:07:45.880934 1105 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:45.881646 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:45.881877 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:45.882633 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:46.061434 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.561s (SENT) I0507 20:07:46.061522 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:46.061902 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.432s (election complete) I0507 20:07:46.062005 1105 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:46.087915 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.661s (SENT) I0507 20:07:46.088013 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:46.088321 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.939s (election complete) I0507 20:07:46.088423 1105 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:46.291755 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.409s (SENT) I0507 20:07:46.291858 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:46.292170 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.818s (election complete) I0507 20:07:46.292263 1105 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:46.471438 1105 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:46.471549 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.416s (starting election) I0507 20:07:46.471626 1105 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:46.472057 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:46.472242 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:46.473500 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:46.612864 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.860s (SENT) I0507 20:07:46.613016 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:46.613441 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.266s (election complete) I0507 20:07:46.613531 1105 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:46.815821 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:43.817129 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:43.817404 (+ 275us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:43.817418 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:43.817535 (+ 117us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:46.815390 (+2997855us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":91} W0507 20:07:46.816177 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:46.880862 1105 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:46.880975 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.187s (starting election) I0507 20:07:46.881052 1105 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:46.881475 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:46.881654 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:46.882539 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:46.889575 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.416s (SENT) I0507 20:07:46.889703 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:46.890151 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.561s (election complete) I0507 20:07:46.890245 1105 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:47.028712 1105 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:47.028900 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.557s (starting election) I0507 20:07:47.029115 1105 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:47.029842 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:47.030130 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:47.030983 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:47.069231 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.187s (SENT) I0507 20:07:47.069370 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:47.069777 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.214s (election complete) I0507 20:07:47.069866 1105 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:47.111002 1105 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:47.111182 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.671s (starting election) I0507 20:07:47.111304 1105 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:47.112035 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:47.112331 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:47.113173 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:47.284654 1105 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:47.284813 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.999s (starting election) I0507 20:07:47.284925 1105 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:47.285550 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:47.285804 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:47.286779 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:47.586598 1105 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:47.586707 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.894s (starting election) I0507 20:07:47.586814 1105 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:47.587250 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:47.587411 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:47.588002 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:47.588884 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.557s (SENT) I0507 20:07:47.588930 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:47.589081 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.228s (election complete) I0507 20:07:47.589128 1105 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:47.782953 1105 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:47.783061 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.967s (starting election) I0507 20:07:47.783149 1105 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:47.783587 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:47.783740 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:47.783982 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.671s (SENT) I0507 20:07:47.784024 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:47.784174 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.458s (election complete) I0507 20:07:47.784229 1105 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:47.784426 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:47.817509 1105 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:47.817646 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.253s (starting election) I0507 20:07:47.817739 1105 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:47.818183 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:47.818370 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:47.819093 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:48.071676 1105 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:48.071753 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.310s (starting election) I0507 20:07:48.071822 1105 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:48.072335 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:48.072432 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.253s (SENT) I0507 20:07:48.072484 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:48.072502 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:48.072626 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.109s (election complete) I0507 20:07:48.072660 1105 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:48.073547 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:48.182483 1109 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:48.182612 1109 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.212s (starting election) I0507 20:07:48.182694 1109 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:48.183131 1109 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:48.183281 1109 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:48.184031 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:48.243305 1109 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:48.243408 1109 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.450s (starting election) I0507 20:07:48.243484 1109 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:48.243921 1109 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:48.244091 1109 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:48.244876 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:48.285308 1109 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:48.285478 1109 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.889s (starting election) I0507 20:07:48.285657 1109 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:48.286388 1109 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:48.286406 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.999s (SENT) I0507 20:07:48.286492 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:48.286672 1109 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:48.286715 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.784s (election complete) I0507 20:07:48.286761 1105 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:48.287734 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:48.384702 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.310s (SENT) I0507 20:07:48.384800 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:48.385082 1109 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.648s (election complete) I0507 20:07:48.385162 1109 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:48.396616 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.212s (SENT) I0507 20:07:48.396708 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:48.396965 1109 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.206s (election complete) I0507 20:07:48.397037 1109 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:48.482836 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.894s (SENT) I0507 20:07:48.482946 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:48.483233 1109 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.559s (election complete) I0507 20:07:48.483314 1109 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:48.694648 1109 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:48.694762 1109 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.666s (starting election) I0507 20:07:48.694870 1109 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:48.695335 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.450s (SENT) I0507 20:07:48.695361 1109 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:48.695403 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:48.695534 1109 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:48.695641 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.669s (election complete) I0507 20:07:48.695691 1105 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:48.696525 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:48.752245 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.967s (SENT) I0507 20:07:48.752359 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:48.752635 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.723s (election complete) I0507 20:07:48.752704 1105 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:49.043387 1105 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:49.043509 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.550s (starting election) I0507 20:07:49.043582 1105 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:49.043987 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:49.044126 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:49.044827 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:49.071584 1105 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:49.071702 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.898s (starting election) I0507 20:07:49.071794 1105 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:49.072373 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:49.072571 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:49.073410 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:49.176559 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.889s (SENT) I0507 20:07:49.176659 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:49.176964 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.495s (election complete) I0507 20:07:49.177052 1105 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:49.362831 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.666s (SENT) I0507 20:07:49.362941 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:49.363281 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.079s (election complete) I0507 20:07:49.363369 1105 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:49.443519 1105 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:49.443626 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.566s (starting election) I0507 20:07:49.443707 1105 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:49.444146 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:49.444308 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:49.445000 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:49.593957 1105 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:49.594076 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.832s (starting election) I0507 20:07:49.594146 1105 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:49.594559 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:49.594738 1105 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:49.595023 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.550s (SENT) I0507 20:07:49.595077 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:49.595278 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.515s (election complete) I0507 20:07:49.595315 1105 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:49.595453 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:49.673301 1105 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:49.673409 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.360s (starting election) I0507 20:07:49.673501 1105 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:49.673923 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:49.674150 1105 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:49.674860 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:49.972177 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.898s (SENT) I0507 20:07:49.972263 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:49.972537 1105 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.188s (election complete) I0507 20:07:49.972623 1105 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:50.010756 1105 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:50.010954 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.668s (starting election) I0507 20:07:50.011062 1105 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:50.011632 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:50.011667 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.566s (SENT) I0507 20:07:50.011723 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:50.011829 1105 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:50.011958 1105 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.625s (election complete) I0507 20:07:50.011999 1105 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:50.012578 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:50.035316 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.360s (SENT) I0507 20:07:50.035437 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:50.035730 1110 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.901s (election complete) I0507 20:07:50.035815 1110 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:50.111495 1110 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:50.111591 1110 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.217s (starting election) I0507 20:07:50.111661 1110 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:50.112047 1110 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:50.112211 1110 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:50.112844 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:50.195451 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:47.194461 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:47.194755 (+ 294us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:47.194770 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:47.194859 (+ 89us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:50.195160 (+3000301us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":98} W0507 20:07:50.195762 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:50.329879 1110 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:50.329928 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.217s (SENT) I0507 20:07:50.329982 1110 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.862s (starting election) I0507 20:07:50.330005 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:50.330039 1110 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:50.330622 1110 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:50.330641 1105 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.292s (election complete) I0507 20:07:50.330703 1105 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:50.330793 1110 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:50.331490 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:50.428784 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.832s (SENT) I0507 20:07:50.428877 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:50.429180 1110 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.929s (election complete) I0507 20:07:50.429256 1110 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:50.638008 1110 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:50.638134 1110 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.241s (starting election) I0507 20:07:50.638238 1110 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:50.638686 1110 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:50.638846 1110 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:50.639600 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:50.681566 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.668s (SENT) I0507 20:07:50.681671 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:50.681998 1110 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.175s (election complete) I0507 20:07:50.682098 1110 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:50.857762 1110 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:50.857872 1110 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.794s (starting election) I0507 20:07:50.857951 1110 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:50.858409 1110 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:50.858566 1110 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:50.859268 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:50.880416 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.241s (SENT) I0507 20:07:50.880519 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:50.880786 1110 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.150s (election complete) I0507 20:07:50.880851 1110 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:50.937252 1110 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:50.937346 1110 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.113s (starting election) I0507 20:07:50.937422 1110 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:50.937809 1110 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:50.937968 1110 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:50.938683 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:51.031574 1110 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:51.031685 1110 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.662s (starting election) I0507 20:07:51.031770 1110 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:51.032253 1110 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:51.032475 1110 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:51.033216 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:51.051106 1110 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:51.051164 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.113s (SENT) I0507 20:07:51.051199 1110 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.613s (starting election) I0507 20:07:51.051230 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:51.051264 1110 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:51.051640 1110 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:51.051645 1113 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.969s (election complete) I0507 20:07:51.051686 1113 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:51.051772 1110 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:51.052471 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:51.194074 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.862s (SENT) I0507 20:07:51.194185 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:51.194501 1110 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.726s (election complete) I0507 20:07:51.194597 1110 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:51.653162 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.794s (SENT) I0507 20:07:51.653301 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:51.653611 1110 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.616s (election complete) I0507 20:07:51.653688 1110 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:51.665917 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.613s (SENT) I0507 20:07:51.666090 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:51.666436 1110 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.523s (election complete) I0507 20:07:51.666524 1110 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:51.695845 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.662s (SENT) I0507 20:07:51.695958 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:51.696295 1110 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.408s (election complete) I0507 20:07:51.696384 1110 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:51.921407 1110 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:51.921573 1110 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.181s (starting election) I0507 20:07:51.921685 1110 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:51.922238 1110 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:51.922437 1110 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:51.923579 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:52.104022 1110 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:52.104158 1110 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.908s (starting election) I0507 20:07:52.104265 1110 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:07:52.104696 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.181s (SENT) I0507 20:07:52.104763 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:52.104846 1110 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:52.105036 1110 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:52.105152 1110 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.370s (election complete) I0507 20:07:52.105185 1110 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:52.105219 1110 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:52.105237 1110 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.161s (starting election) I0507 20:07:52.105268 1110 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:52.105535 1110 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:52.105634 1110 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:52.105823 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:52.105918 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:52.180935 1116 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:52.181100 1116 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.706s (starting election) I0507 20:07:52.181223 1116 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:52.181733 1116 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:52.181864 1116 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:52.182508 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:52.190760 1116 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:52.190860 1116 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.660s (starting election) I0507 20:07:52.190954 1116 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:52.191380 1116 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:52.191529 1116 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:52.192294 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:52.267828 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.161s (SENT) I0507 20:07:52.267920 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:52.268235 1116 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.800s (election complete) I0507 20:07:52.268307 1116 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:52.476261 1116 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:52.476377 1116 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.727s (starting election) I0507 20:07:52.476457 1116 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:52.476943 1116 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:52.477108 1116 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:52.477864 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:52.851591 1116 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:52.851712 1116 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.169s (starting election) I0507 20:07:52.851783 1116 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:52.852217 1116 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:52.852397 1116 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:52.852622 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.660s (SENT) I0507 20:07:52.852687 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:52.852892 1116 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.641s (election complete) I0507 20:07:52.852974 1116 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:52.853308 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:52.888919 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.706s (SENT) I0507 20:07:52.889036 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:52.889351 1116 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.736s (election complete) I0507 20:07:52.889444 1116 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:53.015214 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.908s (SENT) I0507 20:07:53.015316 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:53.015574 1116 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.729s (election complete) I0507 20:07:53.015650 1116 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:53.022037 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.169s (SENT) I0507 20:07:53.022140 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:53.022402 1116 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.092s (election complete) I0507 20:07:53.022470 1116 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:53.115214 1116 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:53.115329 1116 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.279s (starting election) I0507 20:07:53.115417 1116 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:53.115926 1116 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:53.116112 1116 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:53.116891 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:53.205489 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.727s (SENT) I0507 20:07:53.205576 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:53.205816 1116 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.923s (election complete) I0507 20:07:53.205884 1116 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:53.394953 1116 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:53.395061 1116 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.825s (starting election) I0507 20:07:53.395129 1116 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:53.395542 1116 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:53.395694 1116 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:53.395850 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.279s (SENT) I0507 20:07:53.395917 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:53.396061 1116 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.933s (election complete) I0507 20:07:53.396111 1116 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:53.396391 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:53.579567 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:50.579713 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:50.579962 (+ 249us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:50.579976 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:50.580116 (+ 140us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:53.579239 (+2999123us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":72} W0507 20:07:53.579869 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:53.626369 1116 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:53.626480 1116 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.346s (starting election) I0507 20:07:53.626570 1116 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:53.627087 1116 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:53.627246 1116 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:53.627988 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:53.973058 1116 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:53.973158 1116 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.203s (starting election) I0507 20:07:53.973232 1116 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:53.973811 1116 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:53.973976 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.346s (SENT) I0507 20:07:53.974071 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:53.974063 1116 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:53.975042 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:53.975232 1116 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.222s (election complete) I0507 20:07:53.975291 1116 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:54.130172 1116 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:54.130286 1116 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.968s (starting election) I0507 20:07:54.130363 1116 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:54.130790 1116 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:54.130960 1116 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:54.131603 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:54.178310 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.203s (SENT) I0507 20:07:54.178412 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:54.178699 1116 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.113s (election complete) I0507 20:07:54.178766 1116 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:54.221770 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.825s (SENT) I0507 20:07:54.221859 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:54.222101 1116 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.614s (election complete) I0507 20:07:54.222162 1116 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:54.293119 1116 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:54.293233 1116 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.804s (starting election) I0507 20:07:54.293306 1116 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:54.293736 1116 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:54.293908 1116 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:54.294615 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:54.837077 1119 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:54.837169 1119 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.857s (starting election) I0507 20:07:54.837241 1119 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:54.837658 1119 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:54.837821 1119 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:54.838830 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:55.097923 1119 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:55.098042 1119 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.727s (starting election) I0507 20:07:55.098119 1119 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:55.098529 1119 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:55.098688 1119 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:55.098978 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.804s (SENT) I0507 20:07:55.099067 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:55.099213 1119 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (election complete) I0507 20:07:55.099246 1119 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:55.099416 1119 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:07:55.099452 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.968s (SENT) W0507 20:07:55.099447 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:55.099489 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:55.099465 1119 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.485s (starting election) I0507 20:07:55.099597 1119 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:55.099864 1119 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:55.099952 1119 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:55.100026 1119 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.429s (election complete) I0507 20:07:55.100052 1119 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:55.100455 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:55.421288 1121 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:55.421422 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.585s (starting election) I0507 20:07:55.421507 1121 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:55.421955 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:55.422122 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:55.422807 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:55.529449 1121 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:55.529573 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.198s (starting election) I0507 20:07:55.529685 1121 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:55.530190 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:55.530362 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:55.531196 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:55.586231 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.485s (SENT) I0507 20:07:55.586324 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:55.586657 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.077s (election complete) I0507 20:07:55.586720 1121 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:55.664940 1121 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:55.665071 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.267s (starting election) I0507 20:07:55.665187 1121 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:55.665647 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:55.665848 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:55.667475 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:55.695233 1121 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:55.695370 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.951s (starting election) I0507 20:07:55.695466 1121 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:55.695938 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:55.696117 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:55.696347 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.857s (SENT) I0507 20:07:55.696420 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:55.696609 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.272s (election complete) I0507 20:07:55.696714 1121 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:55.696931 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:55.729387 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.198s (SENT) I0507 20:07:55.729506 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:55.729797 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.714s (election complete) I0507 20:07:55.729873 1121 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:55.827421 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.727s (SENT) I0507 20:07:55.827519 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:55.827812 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.204s (election complete) I0507 20:07:55.827934 1121 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:55.934041 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.267s (SENT) I0507 20:07:55.934164 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:55.934489 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.628s (election complete) I0507 20:07:55.934581 1121 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:55.969614 1121 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:55.969755 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.636s (starting election) I0507 20:07:55.969867 1121 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:55.970332 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:55.970521 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:55.971397 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:56.009668 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.585s (SENT) I0507 20:07:56.009774 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:56.010036 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.316s (election complete) I0507 20:07:56.010135 1121 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:56.326704 1121 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:56.326843 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.209s (starting election) I0507 20:07:56.326975 1121 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:56.327399 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:56.327569 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:56.328282 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:56.536536 1121 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:56.536662 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.164s (starting election) I0507 20:07:56.536789 1121 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:56.537312 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:56.537510 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:56.537569 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.209s (SENT) I0507 20:07:56.537714 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:56.537912 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.371s (election complete) I0507 20:07:56.537979 1121 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:56.538239 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:56.563307 1121 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:56.563433 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.177s (starting election) I0507 20:07:56.563539 1121 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:56.564074 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:56.564265 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:56.565016 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:56.607028 1121 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:56.607167 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.550s (starting election) I0507 20:07:56.607283 1121 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:56.607861 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:56.608064 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:56.608103 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.636s (SENT) I0507 20:07:56.608608 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:56.608827 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.102s (election complete) W0507 20:07:56.608860 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:56.608898 1121 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:56.648296 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.951s (SENT) I0507 20:07:56.648411 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:56.648725 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.709s (election complete) I0507 20:07:56.648823 1121 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:56.702373 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.164s (SENT) I0507 20:07:56.702479 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:56.702788 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.310s (election complete) I0507 20:07:56.702911 1121 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:56.741883 1121 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:56.742017 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.712s (starting election) W0507 20:07:56.742241 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.177s (SENT) I0507 20:07:56.742131 1121 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:56.742329 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:56.742720 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:56.742833 1123 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.079s (election complete) I0507 20:07:56.742926 1123 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:56.742959 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:56.743579 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:56.822567 1121 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:56.822681 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.660s (starting election) I0507 20:07:56.822770 1121 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:56.823223 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:56.823385 1121 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:56.824128 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:56.975431 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:53.976643 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:53.976877 (+ 234us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:53.976891 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:53.976988 (+ 97us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:07:56.975171 (+2998183us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":79} W0507 20:07:56.975706 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:07:57.014232 1121 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:57.014369 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.458s (starting election) I0507 20:07:57.014478 1121 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:57.015075 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:57.015292 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:57.016021 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:57.160199 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.550s (SENT) I0507 20:07:57.160310 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:57.160701 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.191s (election complete) I0507 20:07:57.160780 1121 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:57.353006 1121 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:57.353153 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.260s (starting election) I0507 20:07:57.353248 1121 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:57.353700 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:57.353839 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:57.354665 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:57.456316 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.712s (SENT) I0507 20:07:57.456421 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:57.456822 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.521s (election complete) I0507 20:07:57.456909 1121 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:57.472759 1121 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:57.472869 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.317s (starting election) I0507 20:07:57.472954 1121 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:57.473364 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:57.473528 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:57.473865 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.458s (SENT) I0507 20:07:57.473933 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:57.474149 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.139s (election complete) I0507 20:07:57.474227 1121 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:57.474259 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:57.484567 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.660s (SENT) I0507 20:07:57.484642 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:57.484908 1121 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.815s (election complete) I0507 20:07:57.484974 1121 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:57.613896 1121 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:57.613989 1121 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (starting election) I0507 20:07:57.614066 1121 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:57.614514 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:57.614661 1121 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:57.614737 1121 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:57.614778 1121 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.823s (starting election) I0507 20:07:57.614823 1121 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:57.615108 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:57.615206 1121 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:57.615375 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:57.615433 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:57.615749 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.260s (SENT) I0507 20:07:57.615805 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:57.615948 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.537s (election complete) I0507 20:07:57.615988 1126 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:57.791484 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.317s (SENT) I0507 20:07:57.791577 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:57.791844 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.161s (election complete) I0507 20:07:57.791913 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:57.954030 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:57.954172 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.724s (starting election) I0507 20:07:57.954269 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:57.954813 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:57.955041 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:57.955766 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:58.154038 1126 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:58.154165 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.910s (starting election) I0507 20:07:58.154260 1126 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:58.154767 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:58.154975 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:58.155748 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:58.301053 1126 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:58.301206 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.609s (starting election) I0507 20:07:58.301307 1126 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:58.301869 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:58.302079 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:58.302790 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:58.332370 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.716s (SENT) I0507 20:07:58.332470 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:58.332844 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.296s (election complete) I0507 20:07:58.332901 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:58.440568 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.823s (SENT) I0507 20:07:58.440680 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:58.440974 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.290s (election complete) I0507 20:07:58.441045 1126 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:58.629667 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:58.629778 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.420s (starting election) I0507 20:07:58.629863 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:58.630304 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:58.630461 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:58.631203 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:58.679941 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.724s (SENT) I0507 20:07:58.680063 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:58.680392 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.386s (election complete) I0507 20:07:58.680462 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:58.732228 1126 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:58.732326 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.958s (starting election) I0507 20:07:58.732403 1126 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:58.732883 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:58.733037 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:58.733839 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:58.911164 1126 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:58.911285 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.293s (starting election) I0507 20:07:58.911362 1126 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:58.911824 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:58.911988 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:58.912283 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.609s (SENT) I0507 20:07:58.912348 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:58.912796 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.573s (election complete) W0507 20:07:58.912786 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:58.912847 1126 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:59.051086 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.420s (SENT) I0507 20:07:59.051182 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:59.051479 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.528s (election complete) I0507 20:07:59.051561 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:07:59.066617 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.910s (SENT) I0507 20:07:59.066733 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:59.067041 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.753s (election complete) I0507 20:07:59.067116 1126 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:07:59.205842 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.293s (SENT) I0507 20:07:59.205955 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:59.206290 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.269s (election complete) I0507 20:07:59.206357 1126 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:59.475695 1126 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:59.475792 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.394s (starting election) I0507 20:07:59.475860 1126 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:59.476263 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:59.476388 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:59.477895 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:59.580967 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:59.581107 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.415s (starting election) I0507 20:07:59.581212 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:59.581804 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:59.582018 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:59.582762 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:07:59.692865 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.958s (SENT) I0507 20:07:59.692966 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:59.693280 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.202s (election complete) I0507 20:07:59.693358 1126 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:59.870527 1126 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:59.870638 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.229s (starting election) I0507 20:07:59.870718 1126 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:59.871192 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:59.871448 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.394s (SENT) I0507 20:07:59.871506 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:07:59.871966 1128 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.510s (election complete) I0507 20:07:59.872030 1128 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:07:59.872231 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:59.872896 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:59.895957 1126 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:59.896066 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.327s (starting election) I0507 20:07:59.896157 1126 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:07:59.896613 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:07:59.896772 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:07:59.897526 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:07:59.997005 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:07:59.997108 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.122s (starting election) W0507 20:07:59.997244 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.415s (SENT) I0507 20:07:59.997179 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:07:59.997323 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:07:59.997555 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:07:59.997577 1128 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.551s (election complete) I0507 20:07:59.997632 1128 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:07:59.997686 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:07:59.998543 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:00.102264 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.229s (SENT) I0507 20:08:00.102370 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:00.102663 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.589s (election complete) I0507 20:08:00.102733 1126 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:00.120942 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.122s (SENT) I0507 20:08:00.121039 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:00.121331 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.671s (election complete) I0507 20:08:00.121428 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:00.223686 1126 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:00.223789 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.267s (starting election) I0507 20:08:00.223862 1126 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:00.224269 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:00.224455 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:00.224654 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.327s (SENT) I0507 20:08:00.224709 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:00.224874 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.769s (election complete) I0507 20:08:00.224927 1126 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:00.225160 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:00.379492 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:07:57.381595 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:07:57.381831 (+ 236us) negotiation.cc:98] Waiting for socket to connect 0507 20:07:57.381844 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:07:57.381928 (+ 84us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:00.379176 (+2997248us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":59} W0507 20:08:00.379770 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:08:00.492447 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.267s (SENT) I0507 20:08:00.492547 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:00.492837 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.550s (election complete) I0507 20:08:00.492924 1126 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:00.692096 1126 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:00.692210 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.453s (starting election) I0507 20:08:00.692288 1126 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:00.692760 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:00.692912 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:00.693631 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:00.792994 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:00.793109 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.233s (starting election) I0507 20:08:00.793198 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:00.793716 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:00.793885 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:00.794487 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:01.026448 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:01.026554 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.113s (starting election) I0507 20:08:01.026629 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:01.027084 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:01.027254 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:01.027534 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.233s (SENT) I0507 20:08:01.027606 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:01.027784 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.983s (election complete) I0507 20:08:01.027820 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:01.028007 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:01.043603 1126 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:01.043705 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.953s (starting election) I0507 20:08:01.043779 1126 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:01.044149 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:01.044260 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:01.045042 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:01.141036 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.113s (SENT) I0507 20:08:01.141130 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:01.141423 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.144s (election complete) I0507 20:08:01.141497 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:01.145486 1126 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:01.145581 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.398s (starting election) I0507 20:08:01.145655 1126 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:01.146065 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:01.146220 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:01.146497 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.453s (SENT) I0507 20:08:01.146553 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:01.146731 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.715s (election complete) I0507 20:08:01.146783 1126 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:01.146858 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:01.286715 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:01.286844 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.095s (starting election) I0507 20:08:01.287020 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:01.287564 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:01.287778 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:01.288465 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:01.382632 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:01.382741 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.732s (starting election) I0507 20:08:01.382815 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:01.383268 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:01.383446 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:01.383738 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.095s (SENT) I0507 20:08:01.383797 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:01.383945 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.343s (election complete) I0507 20:08:01.384011 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:01.384205 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:01.544833 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.398s (SENT) I0507 20:08:01.544958 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:01.545270 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.186s (election complete) I0507 20:08:01.545361 1126 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:01.727577 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:01.727717 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.365s (starting election) I0507 20:08:01.727862 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:01.728426 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:01.728579 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:01.729300 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:01.731729 1126 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:01.731801 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.406s (starting election) I0507 20:08:01.731885 1126 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:01.732306 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:01.732450 1126 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:01.733134 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:01.997931 1126 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:01.998057 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.223s (starting election) I0507 20:08:01.998157 1126 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:01.998703 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:01.998942 1126 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:01.999161 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.953s (SENT) I0507 20:08:01.999225 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:01.999413 1126 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.168s (election complete) I0507 20:08:01.999461 1126 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:01.999773 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:02.094151 1126 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:02.094260 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.860s (starting election) I0507 20:08:02.094362 1126 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:02.094763 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.365s (SENT) I0507 20:08:02.094815 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:02.094915 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:02.095041 1126 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:02.095125 1126 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.535s (election complete) I0507 20:08:02.095155 1126 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:02.095770 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:02.117130 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.732s (SENT) I0507 20:08:02.117254 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:02.117621 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.135s (election complete) I0507 20:08:02.117689 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:02.138634 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:02.138763 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.256s (starting election) I0507 20:08:02.138870 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:02.139160 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.406s (SENT) I0507 20:08:02.139233 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:02.139452 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:02.139458 1126 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.968s (election complete) I0507 20:08:02.139503 1126 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:02.139576 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:02.140287 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:02.167886 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:02.167997 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.963s (starting election) I0507 20:08:02.168084 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:02.168601 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:02.168738 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:02.169636 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:02.222738 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.223s (SENT) I0507 20:08:02.222851 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:02.223167 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.972s (election complete) I0507 20:08:02.223249 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:02.253518 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:02.253624 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.504s (starting election) I0507 20:08:02.253708 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:02.254235 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:02.254406 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:02.255409 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:02.396260 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.256s (SENT) I0507 20:08:02.396369 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:02.396653 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.800s (election complete) I0507 20:08:02.396720 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:02.758673 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:02.758822 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.607s (starting election) I0507 20:08:02.758939 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:02.759431 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:02.759620 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:02.759945 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.504s (SENT) I0507 20:08:02.760028 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:02.760228 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.512s (election complete) I0507 20:08:02.760293 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:02.760951 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:02.956553 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.860s (SENT) I0507 20:08:02.956671 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:02.956997 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.897s (election complete) I0507 20:08:02.957109 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:03.132982 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.963s (SENT) I0507 20:08:03.133078 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:03.133347 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.676s (election complete) I0507 20:08:03.133443 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:03.198179 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:03.198303 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.738s (starting election) I0507 20:08:03.198407 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:03.198915 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:03.199113 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:03.200001 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:03.368332 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.607s (SENT) I0507 20:08:03.368459 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:03.368798 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.710s (election complete) I0507 20:08:03.368917 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:03.791565 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:00.793991 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:00.794225 (+ 234us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:00.794237 (+ 12us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:00.794325 (+ 88us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:03.791255 (+2996930us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":75} W0507 20:08:03.791923 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:03.810551 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:03.810703 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.503s (starting election) I0507 20:08:03.810817 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:03.811386 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:03.811612 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:03.812896 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:03.936666 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:03.936817 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.819s (starting election) I0507 20:08:03.936980 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:03.937656 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:03.937882 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:03.938340 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.738s (SENT) I0507 20:08:03.938449 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:08:03.938712 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:03.938786 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.420s (election complete) I0507 20:08:03.938911 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:04.079617 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:04.079716 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.465s (starting election) I0507 20:08:04.079813 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:04.080343 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:04.080499 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:04.081445 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:04.314733 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:04.314959 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.400s (starting election) I0507 20:08:04.315104 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:04.315788 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:04.315783 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.503s (SENT) I0507 20:08:04.315922 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:04.316079 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:04.316303 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.551s (election complete) I0507 20:08:04.316385 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:04.317539 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:04.359226 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:04.359412 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.539s (starting election) I0507 20:08:04.359547 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:04.360297 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:04.360546 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:04.361485 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:04.545895 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:04.546072 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.891s (starting election) I0507 20:08:04.546211 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:04.546571 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.465s (SENT) I0507 20:08:04.546667 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:04.546950 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:04.547013 1136 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.753s (election complete) I0507 20:08:04.547107 1136 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:04.547245 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:04.548017 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:04.717005 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.400s (SENT) I0507 20:08:04.717108 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:04.717530 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.725s (election complete) I0507 20:08:04.717667 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:04.758334 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.819s (SENT) I0507 20:08:04.758440 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:04.758792 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.330s (election complete) I0507 20:08:04.758919 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:04.901306 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.539s (SENT) I0507 20:08:04.901410 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:04.901801 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.888s (election complete) I0507 20:08:04.901911 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:05.301198 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:05.301353 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.296s (starting election) I0507 20:08:05.301462 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:05.301961 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:05.302175 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:05.303403 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:05.440408 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.891s (SENT) I0507 20:08:05.440515 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:05.440897 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.492s (election complete) I0507 20:08:05.441016 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:05.443207 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:05.443281 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.279s (starting election) I0507 20:08:05.443392 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:05.443845 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:05.444027 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:05.444810 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:05.599350 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.296s (SENT) I0507 20:08:05.599483 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:05.599833 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.900s (election complete) I0507 20:08:05.599949 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:05.723486 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.279s (SENT) I0507 20:08:05.723453 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:05.723631 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:05.723739 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.348s (starting election) I0507 20:08:05.723906 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:05.724578 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:05.724591 1138 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.154s (election complete) I0507 20:08:05.724665 1138 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:05.724831 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:05.725805 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:05.790289 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:05.790434 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.474s (starting election) I0507 20:08:05.790555 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:05.791174 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:05.791394 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:05.792361 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:05.879911 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:05.880061 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.535s (starting election) I0507 20:08:05.880175 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:05.880642 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:05.880878 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:05.881803 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:06.073657 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.348s (SENT) I0507 20:08:06.073770 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:06.074093 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.478s (election complete) I0507 20:08:06.074192 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:06.265329 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:06.265472 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.560s (starting election) I0507 20:08:06.265594 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:06.266012 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.474s (SENT) I0507 20:08:06.266109 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:06.266189 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:06.266479 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:06.266664 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.900s (election complete) I0507 20:08:06.266731 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:06.267390 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:06.417165 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.535s (SENT) I0507 20:08:06.417284 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:06.417654 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.946s (election complete) I0507 20:08:06.417762 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:06.500911 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:06.501085 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.364s (starting election) I0507 20:08:06.501206 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:06.501705 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:06.501920 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:06.502918 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:06.827843 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.560s (SENT) I0507 20:08:06.827955 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:06.828356 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.123s (election complete) I0507 20:08:06.828444 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:06.866120 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:06.866263 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.892s (starting election) I0507 20:08:06.866370 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:06.866847 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:06.867089 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:06.867473 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.364s (SENT) I0507 20:08:06.867559 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:06.867794 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.693s (election complete) I0507 20:08:06.867885 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:06.868098 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:06.952564 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:06.952695 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.101s (starting election) I0507 20:08:06.952800 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:06.953284 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:06.953487 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:06.954407 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:07.054697 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:07.054816 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.975s (starting election) I0507 20:08:07.054944 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:07.055402 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:07.055593 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:07.056046 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.101s (SENT) I0507 20:08:07.056154 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:07.056447 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.515s (election complete) W0507 20:08:07.056458 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:07.056535 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:07.219524 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:04.219674 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:04.219940 (+ 266us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:04.219954 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:04.220071 (+ 117us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:07.219220 (+2999149us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":85} W0507 20:08:07.219825 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:07.364292 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:07.364430 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.569s (starting election) I0507 20:08:07.364542 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:07.365070 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:07.365262 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:07.366099 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:07.562275 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:07.562427 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.491s (starting election) I0507 20:08:07.562541 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:07.563161 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:07.564298 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:07.564610 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:07.572306 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:07.572448 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.349s (starting election) I0507 20:08:07.572546 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:07.573035 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:07.573215 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:07.573976 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:07.761355 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.892s (SENT) I0507 20:08:07.761463 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:07.761849 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.720s (election complete) I0507 20:08:07.761958 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:07.922808 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:07.922976 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.532s (starting election) W0507 20:08:07.923207 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.349s (SENT) I0507 20:08:07.923094 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:07.923308 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:07.923672 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:07.923825 1142 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.434s (election complete) I0507 20:08:07.923913 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:07.923930 1142 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:07.924818 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:07.934206 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:07.934319 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.407s (starting election) I0507 20:08:07.934418 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:07.934859 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:07.935045 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:07.935858 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:07.936497 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.569s (SENT) I0507 20:08:07.936579 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:07.936813 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.293s (election complete) I0507 20:08:07.937662 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:08.032114 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.975s (SENT) I0507 20:08:08.032232 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:08.032565 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.323s (election complete) I0507 20:08:08.032672 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:08.055748 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.491s (SENT) I0507 20:08:08.055871 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:08.056187 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.081s (election complete) I0507 20:08:08.056265 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:08.138264 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:08.138408 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.810s (starting election) I0507 20:08:08.138496 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:08.138995 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:08.139179 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:08.140177 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:08.230360 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:08.230491 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.192s (starting election) I0507 20:08:08.230592 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:08.231106 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:08.231293 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:08.232292 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:08.342793 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.407s (SENT) I0507 20:08:08.342932 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:08.343322 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.678s (election complete) I0507 20:08:08.343425 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:08.356374 1131 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:08.356516 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.233s (starting election) I0507 20:08:08.356638 1131 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:08.357281 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:08.357507 1131 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:08.358491 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:08.423991 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.192s (SENT) I0507 20:08:08.424120 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:08.424445 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.094s (election complete) I0507 20:08:08.424542 1131 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:08.457566 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.532s (SENT) I0507 20:08:08.457693 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:08.457965 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.238s (election complete) I0507 20:08:08.458052 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:08.519474 1131 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:08.519610 1131 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.499s (starting election) I0507 20:08:08.519713 1131 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:08.528333 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:08.528548 1131 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:08.529431 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:08.591902 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.233s (SENT) I0507 20:08:08.592042 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:08.592483 1131 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.581s (election complete) I0507 20:08:08.592583 1131 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:08.949415 1131 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:08.949573 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.286s (starting election) I0507 20:08:08.949694 1131 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:08.950387 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:08.950415 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.810s (SENT) I0507 20:08:08.950495 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:08.950626 1131 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:08.950788 1131 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.217s (election complete) I0507 20:08:08.950835 1131 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:08.951536 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:09.020030 1143 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:09.020143 1143 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.405s (starting election) I0507 20:08:09.020220 1143 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:09.020833 1143 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:09.021023 1143 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:09.022130 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:09.027732 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.499s (SENT) I0507 20:08:09.027848 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:09.028239 1143 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.411s (election complete) I0507 20:08:09.028318 1143 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:09.169229 1143 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:09.169342 1143 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.223s (starting election) I0507 20:08:09.169414 1143 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:09.169831 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:09.169972 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:09.171159 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:09.174376 1143 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:09.174499 1143 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.368s (starting election) I0507 20:08:09.174605 1143 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:09.175297 1143 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:09.175528 1143 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:09.176239 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:09.238015 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.286s (SENT) I0507 20:08:09.238129 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:09.238487 1143 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.830s (election complete) I0507 20:08:09.238584 1143 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:09.393580 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.223s (SENT) I0507 20:08:09.393689 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:09.394026 1143 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.198s (election complete) I0507 20:08:09.394124 1143 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:09.426623 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.405s (SENT) I0507 20:08:09.426734 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:09.427106 1143 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.614s (election complete) I0507 20:08:09.427209 1143 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:09.543419 1143 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:09.543552 1143 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.548s (starting election) I0507 20:08:09.543640 1143 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:09.544144 1143 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:09.544337 1143 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:09.544679 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.368s (SENT) I0507 20:08:09.544776 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:09.544994 1143 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.299s (election complete) I0507 20:08:09.545044 1143 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:09.545208 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:09.592793 1143 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:09.592921 1143 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.364s (starting election) I0507 20:08:09.593014 1143 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:09.593506 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:09.593686 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:09.594569 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:09.845090 1143 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:09.845213 1143 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.829s (starting election) I0507 20:08:09.845294 1143 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:09.845762 1143 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:09.845937 1143 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:09.846765 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:09.957664 1143 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:09.957808 1143 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.322s (starting election) W0507 20:08:09.957983 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.364s (SENT) I0507 20:08:09.958067 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:09.957896 1143 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:09.958408 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:09.958575 1144 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.839s (election complete) I0507 20:08:09.958606 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:09.958627 1144 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:09.959357 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:10.041790 1143 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:10.041906 1143 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.407s (starting election) I0507 20:08:10.041983 1143 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:10.042426 1143 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:10.042589 1143 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:10.043514 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:10.093634 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.548s (SENT) I0507 20:08:10.093734 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:10.094043 1143 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.674s (election complete) I0507 20:08:10.094122 1143 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:10.281774 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.322s (SENT) I0507 20:08:10.281867 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:10.282141 1143 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.320s (election complete) I0507 20:08:10.282213 1143 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:10.450242 1143 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:10.450285 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.407s (SENT) I0507 20:08:10.450350 1143 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.308s (starting election) I0507 20:08:10.450373 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:10.450422 1143 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:10.450857 1143 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:10.450907 1144 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.518s (election complete) I0507 20:08:10.450958 1144 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:10.451092 1143 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:10.452067 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:10.603153 1143 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:10.603267 1143 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.380s (starting election) I0507 20:08:10.603351 1143 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:10.603790 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:10.603950 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:10.604959 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:10.651657 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:07.650005 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:07.651257 (+ 1252us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:07.651275 (+ 18us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:07.651408 (+ 133us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:10.651291 (+2999883us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":106} W0507 20:08:10.652017 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:08:10.676501 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.829s (SENT) I0507 20:08:10.676620 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:10.676952 1143 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.827s (election complete) I0507 20:08:10.677029 1143 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:10.759562 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.308s (SENT) I0507 20:08:10.759682 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:10.760035 1143 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.792s (election complete) I0507 20:08:10.760107 1143 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:10.984746 1143 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:10.984855 1143 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.404s (starting election) W0507 20:08:10.984885 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.380s (SENT) I0507 20:08:10.984972 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:10.984926 1143 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:10.985353 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:10.985527 1143 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:10.985540 1146 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.923s (election complete) I0507 20:08:10.985584 1146 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:10.986410 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:11.390820 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.404s (SENT) I0507 20:08:11.390928 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:11.391185 1146 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.897s (election complete) I0507 20:08:11.391258 1146 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:11.504806 1146 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:11.504911 1146 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.304s (starting election) I0507 20:08:11.504990 1146 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:11.505448 1146 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:11.505605 1146 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:11.506614 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:11.552894 1146 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:11.553035 1146 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.933s (starting election) I0507 20:08:11.553122 1146 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:11.553619 1146 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:11.553798 1146 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:11.554769 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:11.810024 1146 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:11.810132 1146 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.723s (starting election) W0507 20:08:11.810322 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.304s (SENT) I0507 20:08:11.810216 1146 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:11.810403 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:11.810699 1146 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:11.810892 1146 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:11.810902 1149 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.171s (election complete) I0507 20:08:11.810945 1149 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:11.811599 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:11.982707 1149 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:11.982837 1149 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.254s (starting election) I0507 20:08:11.982952 1149 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:11.983414 1149 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:11.983589 1149 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:11.984454 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:12.237960 1149 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:12.238080 1149 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.881s (starting election) I0507 20:08:12.238162 1149 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:12.238443 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.254s (SENT) I0507 20:08:12.238548 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:12.238649 1149 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:12.238874 1149 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:12.238859 1146 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.520s (election complete) I0507 20:08:12.238955 1146 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:12.239686 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:12.289502 1149 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:12.289616 1149 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.304s (starting election) I0507 20:08:12.289695 1149 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:12.290104 1149 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:12.290264 1149 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:12.291095 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:12.486455 1149 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:12.486590 1149 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.919s (starting election) I0507 20:08:12.486680 1149 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:12.487174 1149 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:12.487345 1149 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:12.487615 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.933s (SENT) I0507 20:08:12.487700 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:12.487936 1149 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.606s (election complete) I0507 20:08:12.488036 1149 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:12.488178 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:12.534867 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.723s (SENT) I0507 20:08:12.535001 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:12.535324 1149 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.640s (election complete) I0507 20:08:12.535437 1149 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:12.594050 1149 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:12.594163 1149 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.228s (starting election) I0507 20:08:12.594244 1149 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:12.594717 1149 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:12.594743 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.304s (SENT) I0507 20:08:12.594835 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:12.594907 1149 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:12.595080 1149 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.868s (election complete) I0507 20:08:12.595122 1149 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:12.595599 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:12.823319 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.228s (SENT) I0507 20:08:12.823416 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:12.823704 1149 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.344s (election complete) I0507 20:08:12.823781 1149 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:13.094954 1149 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:13.095077 1149 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.910s (starting election) I0507 20:08:13.095176 1149 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:13.095649 1149 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:13.095841 1149 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:13.097559 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:13.121428 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.881s (SENT) I0507 20:08:13.121527 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:13.121860 1149 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.804s (election complete) I0507 20:08:13.121959 1149 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:13.168735 1149 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:13.168841 1149 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.756s (starting election) I0507 20:08:13.168915 1149 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:13.169322 1149 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:13.169468 1149 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:13.170291 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:13.407904 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.919s (SENT) I0507 20:08:13.408007 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:13.408356 1149 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.485s (election complete) I0507 20:08:13.408463 1149 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:13.893846 1149 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:13.893966 1149 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.946s (starting election) I0507 20:08:13.894047 1149 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:13.894516 1149 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:13.894683 1149 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:13.895933 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:13.925528 1149 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:13.925653 1149 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.337s (starting election) I0507 20:08:13.925779 1149 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:13.926268 1149 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:13.926450 1149 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:13.926738 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.756s (SENT) I0507 20:08:13.926791 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:13.926861 1149 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:13.926971 1149 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.540s (starting election) I0507 20:08:13.927047 1149 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:13.927284 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:13.927479 1149 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:13.927600 1149 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:13.927685 1149 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.821s (election complete) I0507 20:08:13.927717 1149 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:13.928002 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:14.006840 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.910s (SENT) I0507 20:08:14.006958 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:14.007272 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.160s (election complete) I0507 20:08:14.007346 1152 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:14.095563 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:11.094708 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:11.095000 (+ 292us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:11.095017 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:11.095130 (+ 113us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:14.095231 (+3000101us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":75} W0507 20:08:14.095870 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:14.168308 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:14.168439 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.982s (starting election) I0507 20:08:14.168527 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:14.169052 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:14.169232 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:14.170081 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:14.264719 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.337s (SENT) I0507 20:08:14.264816 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:14.265144 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.406s (election complete) I0507 20:08:14.265247 1152 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:14.467737 1152 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:14.467844 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.579s (starting election) I0507 20:08:14.467932 1152 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:14.468418 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:14.468576 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:14.468899 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.540s (SENT) I0507 20:08:14.469000 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:08:14.469305 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:14.469256 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.597s (election complete) I0507 20:08:14.469382 1152 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:14.672287 1152 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:14.672432 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.352s (starting election) I0507 20:08:14.672535 1152 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:14.673097 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:14.673288 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:14.674206 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:14.841943 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.946s (SENT) I0507 20:08:14.842037 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:14.842350 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.129s (election complete) I0507 20:08:14.842455 1152 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:14.972528 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:14.972661 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.384s (starting election) I0507 20:08:14.972751 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:14.973254 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:14.973433 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:14.974345 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:15.025166 1152 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:15.025300 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.482s (starting election) I0507 20:08:15.025414 1152 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:15.026077 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:15.026288 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:15.026335 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.352s (SENT) I0507 20:08:15.026437 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:15.026633 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.641s (election complete) I0507 20:08:15.026731 1152 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:15.027190 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:15.048584 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.579s (SENT) I0507 20:08:15.048693 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:15.048990 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.631s (election complete) I0507 20:08:15.049078 1152 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:15.152379 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.982s (SENT) I0507 20:08:15.152518 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:15.152936 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.377s (election complete) I0507 20:08:15.153033 1152 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:15.358660 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.384s (SENT) I0507 20:08:15.358769 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:15.359189 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.585s (election complete) I0507 20:08:15.359306 1152 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:15.509367 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.482s (SENT) I0507 20:08:15.509472 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:15.509869 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.214s (election complete) I0507 20:08:15.510000 1152 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:15.681308 1152 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:15.681427 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.673s (starting election) I0507 20:08:15.681509 1152 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:15.681979 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:15.682148 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:15.683341 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:15.724643 1152 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:15.724804 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.379s (starting election) I0507 20:08:15.724911 1152 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:15.725419 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:15.725592 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:15.726397 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:15.944681 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:15.944804 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.372s (starting election) I0507 20:08:15.944885 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:15.945415 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:15.945590 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:15.946488 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:16.104610 1152 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:16.104724 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.731s (starting election) I0507 20:08:16.104826 1152 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:16.105340 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:16.105526 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:16.105603 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.379s (SENT) I0507 20:08:16.105684 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:16.105933 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.514s (election complete) I0507 20:08:16.105988 1152 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:16.106393 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:16.317586 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:16.317719 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.650s (starting election) I0507 20:08:16.317816 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:16.318311 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:16.318485 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:16.318794 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.372s (SENT) I0507 20:08:16.318929 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:16.319182 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.660s (election complete) I0507 20:08:16.319237 1152 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:16.319399 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:16.355490 1152 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:16.355618 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.503s (starting election) I0507 20:08:16.355705 1152 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:16.356206 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:16.356405 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:16.356873 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.673s (SENT) I0507 20:08:16.356964 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:16.357194 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.256s (election complete) I0507 20:08:16.357246 1152 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:16.357262 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:16.614539 1152 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:16.614717 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.602s (starting election) I0507 20:08:16.614805 1152 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:16.615315 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:16.615475 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:16.616251 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:16.620229 1152 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:16.620331 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.821s (starting election) I0507 20:08:16.620412 1152 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:16.620888 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:16.621090 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:16.621786 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:16.838476 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.731s (SENT) I0507 20:08:16.838600 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:16.838909 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.807s (election complete) I0507 20:08:16.839046 1152 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:16.860158 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.503s (SENT) I0507 20:08:16.860258 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:16.860610 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.722s (election complete) I0507 20:08:16.860715 1152 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:16.970244 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.650s (SENT) I0507 20:08:16.970347 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:16.970721 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.253s (election complete) I0507 20:08:16.970835 1152 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:17.219287 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.602s (SENT) I0507 20:08:17.219399 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:17.219769 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.895s (election complete) I0507 20:08:17.219883 1152 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:17.224496 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:17.224591 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.253s (starting election) I0507 20:08:17.224664 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:17.225106 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:17.225252 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:17.226286 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:17.443627 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.821s (SENT) I0507 20:08:17.443727 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:17.444046 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.336s (election complete) I0507 20:08:17.444146 1152 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:17.478927 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.253s (SENT) I0507 20:08:17.478909 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:17.479027 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:17.479049 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.923s (starting election) I0507 20:08:17.479149 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:17.479643 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:17.479656 1157 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.425s (election complete) I0507 20:08:17.479727 1157 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:17.479837 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:17.480574 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:17.547554 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:14.550154 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:14.550396 (+ 242us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:14.550409 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:14.550510 (+ 101us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:17.547209 (+2996699us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":72} W0507 20:08:17.547824 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:17.780817 1152 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:17.780935 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.734s (starting election) I0507 20:08:17.781018 1152 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:17.781520 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:17.781680 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:17.782510 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:17.905647 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:17.905807 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.199s (starting election) I0507 20:08:17.905925 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:17.906543 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:17.906796 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:17.907783 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:18.105517 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:18.105633 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.957s (starting election) I0507 20:08:18.105717 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:18.106154 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:18.106313 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:18.106544 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.199s (SENT) I0507 20:08:18.106604 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:18.106812 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.279s (election complete) I0507 20:08:18.106866 1152 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:18.107427 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:18.115552 1152 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:18.115654 1152 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.910s (starting election) I0507 20:08:18.115736 1152 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:18.116221 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:18.116387 1152 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:18.117132 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:18.386586 1152 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:18.386711 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.543s (starting election) I0507 20:08:18.386801 1152 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:18.387264 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:18.387423 1152 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:18.388386 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:18.403564 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.923s (SENT) I0507 20:08:18.403656 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:18.403970 1152 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.969s (election complete) I0507 20:08:18.404067 1152 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:18.516206 1152 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:18.516343 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.936s (starting election) I0507 20:08:18.516433 1152 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:18.516846 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.734s (SENT) I0507 20:08:18.516909 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:18.516927 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:18.517089 1152 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:18.517199 1152 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.773s (election complete) I0507 20:08:18.517246 1152 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:18.517850 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:18.931371 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.543s (SENT) I0507 20:08:18.931468 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:18.931804 1160 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.376s (election complete) I0507 20:08:18.931908 1160 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:19.026584 1160 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:19.026697 1160 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.668s (starting election) I0507 20:08:19.026793 1160 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:19.027092 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.910s (SENT) I0507 20:08:19.027190 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.027293 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:19.027456 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:19.027572 1160 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.291s (election complete) I0507 20:08:19.027621 1160 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:19.028476 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:19.063916 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.957s (SENT) I0507 20:08:19.064035 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.064429 1160 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.167s (election complete) I0507 20:08:19.064539 1160 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:19.232825 1160 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:19.232954 1160 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.179s (starting election) I0507 20:08:19.233053 1160 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:19.233495 1160 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:19.233676 1160 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:19.234644 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:19.291566 1160 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:19.291712 1160 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.676s (starting election) I0507 20:08:19.291818 1160 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:19.292300 1160 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:19.292542 1160 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:19.293356 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:19.320035 1160 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:19.320178 1160 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.270s (starting election) I0507 20:08:19.320281 1160 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:19.320737 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:19.320920 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:19.321691 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:19.413190 1160 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:19.413210 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.179s (SENT) I0507 20:08:19.413316 1160 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.294s (starting election) I0507 20:08:19.413434 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.413462 1160 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:19.413914 1160 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:19.413938 1161 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.373s (election complete) I0507 20:08:19.414016 1161 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:19.414075 1160 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:19.414851 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:19.455016 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.936s (SENT) I0507 20:08:19.455121 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.455435 1160 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.827s (election complete) I0507 20:08:19.455533 1160 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:19.591361 1160 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:19.592002 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.270s (SENT) I0507 20:08:19.592084 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.600225 1160 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.380s (starting election) I0507 20:08:19.600378 1160 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:19.600839 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:19.600878 1161 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.879s (election complete) I0507 20:08:19.600953 1161 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:19.601016 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:19.602010 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:19.696804 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.668s (SENT) I0507 20:08:19.696913 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.697227 1160 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.780s (election complete) I0507 20:08:19.697304 1160 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:19.708650 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.294s (SENT) I0507 20:08:19.708761 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.709028 1160 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.603s (election complete) I0507 20:08:19.709102 1160 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:19.970499 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.676s (SENT) I0507 20:08:19.970616 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.970952 1160 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.336s (election complete) I0507 20:08:19.971022 1160 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:19.981534 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.380s (SENT) I0507 20:08:19.981634 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:19.981931 1160 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.312s (election complete) I0507 20:08:19.981993 1160 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:20.295140 1160 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:20.295249 1160 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.326s (starting election) I0507 20:08:20.295325 1160 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:20.295775 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:20.295958 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:20.296944 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:20.308353 1160 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:20.308483 1160 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.629s (starting election) I0507 20:08:20.308573 1160 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:20.309052 1160 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:20.309273 1160 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:20.310071 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:20.313014 1160 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:20.313109 1160 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.929s (starting election) I0507 20:08:20.313181 1160 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:20.313628 1160 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:20.313772 1160 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:20.314447 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:20.622560 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.326s (SENT) I0507 20:08:20.622678 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:20.622692 1160 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:20.622810 1160 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.671s (starting election) I0507 20:08:20.622910 1160 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:20.623414 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:20.623435 1164 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.674s (election complete) I0507 20:08:20.623481 1164 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:20.623559 1160 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:20.624208 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:20.938174 1160 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:20.938293 1160 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.887s (starting election) I0507 20:08:20.938380 1160 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:20.938823 1160 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:20.939033 1160 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:20.939340 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.629s (SENT) I0507 20:08:20.939391 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:20.939604 1160 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.558s (election complete) I0507 20:08:20.939679 1160 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:20.939831 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:21.003590 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:18.004382 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:18.004724 (+ 342us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:18.004742 (+ 18us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:18.004876 (+ 134us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:21.003264 (+2998388us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":112} W0507 20:08:21.003914 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:21.243362 1160 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:21.243440 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.929s (SENT) I0507 20:08:21.243468 1160 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.737s (starting election) I0507 20:08:21.243502 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:21.243535 1160 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:21.243953 1160 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:21.244071 1160 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:21.244158 1160 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.695s (election complete) I0507 20:08:21.244186 1160 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:21.244843 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:21.295042 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.671s (SENT) I0507 20:08:21.295133 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:21.295418 1166 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.903s (election complete) I0507 20:08:21.295492 1166 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:21.498742 1166 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:21.498868 1166 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.432s (starting election) I0507 20:08:21.498999 1166 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:21.499441 1166 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:21.499606 1166 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:21.500308 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:21.827028 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.887s (SENT) I0507 20:08:21.827127 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:21.827425 1166 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.567s (election complete) I0507 20:08:21.827525 1166 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:21.932260 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.432s (SENT) I0507 20:08:21.932371 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:21.932724 1166 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.300s (election complete) I0507 20:08:21.932822 1166 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:21.940418 1166 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:21.940503 1166 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.353s (starting election) I0507 20:08:21.940572 1166 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:21.941009 1166 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:21.941148 1166 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:21.941844 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:21.983356 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.737s (SENT) I0507 20:08:21.983469 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:21.983858 1166 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.398s (election complete) I0507 20:08:21.983974 1166 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:22.199431 1166 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:22.199537 1166 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.125s (starting election) I0507 20:08:22.199610 1166 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:22.200057 1166 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:22.200209 1166 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:22.200863 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:22.233153 1166 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:22.233273 1166 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.077s (starting election) I0507 20:08:22.233352 1166 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:22.233777 1166 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:22.233968 1166 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:22.234839 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:22.294996 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.353s (SENT) I0507 20:08:22.295135 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:22.295512 1166 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.905s (election complete) I0507 20:08:22.295605 1166 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:22.311040 1166 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:22.311147 1166 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.708s (starting election) W0507 20:08:22.311271 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.077s (SENT) I0507 20:08:22.311368 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:22.311260 1166 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:22.311707 1166 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:22.311894 1166 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:22.311992 1166 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.256s (election complete) I0507 20:08:22.312073 1166 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:22.312647 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:22.325471 1168 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:22.325582 1168 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.621s (starting election) W0507 20:08:22.325763 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.125s (SENT) I0507 20:08:22.325650 1168 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:22.325850 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:22.326117 1168 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:22.326118 1166 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.680s (election complete) I0507 20:08:22.326186 1166 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:22.326711 1168 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:22.326951 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:22.569247 1168 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:22.569383 1168 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.711s (starting election) I0507 20:08:22.569480 1168 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:22.570026 1168 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:22.570228 1168 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:22.571214 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:22.949416 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.621s (SENT) I0507 20:08:22.949517 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:22.949899 1168 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.800s (election complete) I0507 20:08:22.950011 1168 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:23.022785 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.708s (SENT) I0507 20:08:23.022902 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:23.023207 1168 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.615s (election complete) I0507 20:08:23.023306 1168 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:23.201654 1168 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:23.201788 1168 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.659s (starting election) I0507 20:08:23.201874 1168 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:23.202303 1168 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:23.202476 1168 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:23.203620 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:23.282937 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.711s (SENT) I0507 20:08:23.283057 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:23.283413 1168 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.683s (election complete) I0507 20:08:23.283514 1168 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:23.750716 1168 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:23.750823 1168 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.200s (starting election) I0507 20:08:23.750900 1168 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:23.751349 1168 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:23.751489 1168 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:23.755122 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:23.861616 1168 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:23.861732 1168 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.249s (starting election) I0507 20:08:23.861838 1168 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:23.862300 1168 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:23.862460 1168 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:23.862720 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.659s (SENT) I0507 20:08:23.862807 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:23.863139 1168 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.959s (election complete) I0507 20:08:23.863198 1168 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:23.863337 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:23.952188 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.200s (SENT) I0507 20:08:23.952239 1168 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:23.952299 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:23.952349 1168 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.201s (starting election) I0507 20:08:23.952448 1168 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:23.952939 1168 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:23.952950 1171 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.728s (election complete) I0507 20:08:23.953037 1171 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:23.953100 1168 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:23.953842 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:23.967468 1171 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:23.967613 1171 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.490s (starting election) I0507 20:08:23.967722 1171 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:23.968178 1171 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:23.968323 1171 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:23.969200 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:24.112390 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.249s (SENT) I0507 20:08:24.112499 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:24.112860 1171 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.717s (election complete) I0507 20:08:24.113003 1171 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:24.155385 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.201s (SENT) I0507 20:08:24.155503 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:24.155855 1171 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.694s (election complete) I0507 20:08:24.155956 1171 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:24.458644 1171 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:24.458775 1171 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.272s (starting election) W0507 20:08:24.459082 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.490s (SENT) I0507 20:08:24.458957 1171 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:24.459179 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:24.459533 1171 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:24.459748 1171 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:24.459908 1171 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.999s (election complete) I0507 20:08:24.459967 1171 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:24.460572 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:24.471490 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:21.471327 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:21.471739 (+ 412us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:21.471762 (+ 23us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:21.471921 (+ 159us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:24.471207 (+2999286us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":116} W0507 20:08:24.471808 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:08:24.732499 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.272s (SENT) I0507 20:08:24.732673 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:24.733098 1172 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.278s (election complete) I0507 20:08:24.733234 1172 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:24.831471 1172 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:24.831594 1172 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.883s (starting election) I0507 20:08:24.831676 1172 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:24.832223 1172 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:24.832401 1172 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:24.833278 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:24.850366 1172 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:24.850502 1172 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.514s (starting election) I0507 20:08:24.850594 1172 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:24.851120 1172 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:24.851312 1172 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:24.852143 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:25.011801 1172 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:25.011927 1172 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.096s (starting election) I0507 20:08:25.012013 1172 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:25.012470 1172 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:25.012647 1172 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:25.013694 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:25.108830 1172 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:25.108974 1172 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.153s (starting election) I0507 20:08:25.109050 1172 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:25.109226 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.096s (SENT) I0507 20:08:25.109305 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:25.109472 1172 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:25.109621 1172 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:25.109719 1172 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.738s (election complete) I0507 20:08:25.109753 1172 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:25.110607 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:25.263905 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.153s (SENT) I0507 20:08:25.264030 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:25.264425 1175 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.711s (election complete) I0507 20:08:25.264513 1175 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:25.365264 1175 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:25.365412 1175 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.781s (starting election) I0507 20:08:25.365496 1175 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:25.365934 1175 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:25.366075 1175 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:25.366956 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:25.367719 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.514s (SENT) I0507 20:08:25.367790 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:25.368000 1175 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.996s (election complete) I0507 20:08:25.368057 1175 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:25.715509 1175 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:25.715728 1175 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.174s (starting election) I0507 20:08:25.715827 1175 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:25.716406 1175 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:25.716601 1175 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:25.716845 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.883s (SENT) I0507 20:08:25.716907 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:25.717089 1175 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.450s (election complete) I0507 20:08:25.717159 1175 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:25.717715 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:25.891701 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.174s (SENT) I0507 20:08:25.891795 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:25.892143 1175 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.849s (election complete) I0507 20:08:25.892227 1175 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:25.976785 1175 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:25.976907 1175 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.995s (starting election) I0507 20:08:25.976994 1175 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:25.977495 1175 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:25.977715 1175 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:25.978526 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:26.149042 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.781s (SENT) I0507 20:08:26.149166 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:26.149507 1175 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.677s (election complete) I0507 20:08:26.149610 1175 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:26.742758 1176 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:26.742858 1176 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.871s (starting election) I0507 20:08:26.742991 1176 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:26.743453 1176 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:26.743640 1176 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:26.745067 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:26.827420 1176 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:26.827558 1176 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.356s (starting election) I0507 20:08:26.827652 1176 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:26.828096 1176 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:26.828274 1176 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:26.829277 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:26.972623 1176 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:26.972743 1176 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.263s (starting election) I0507 20:08:26.972838 1176 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:26.973333 1176 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:26.973520 1176 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:26.974622 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:26.975177 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.995s (SENT) I0507 20:08:26.975241 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:26.975450 1176 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.620s (election complete) I0507 20:08:26.975546 1176 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:27.184463 1176 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:27.184612 1176 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.583s (starting election) I0507 20:08:27.184710 1176 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:27.185179 1176 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:27.185386 1176 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:27.185497 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.356s (SENT) I0507 20:08:27.185600 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:27.185880 1176 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (election complete) I0507 20:08:27.185986 1176 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:27.186239 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:27.237257 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.263s (SENT) I0507 20:08:27.237376 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:27.237789 1176 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.597s (election complete) I0507 20:08:27.237938 1176 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:27.615097 1176 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:27.615195 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.871s (SENT) I0507 20:08:27.615223 1176 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.237s (starting election) I0507 20:08:27.615274 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:27.615312 1176 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:27.615761 1176 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:27.615844 1178 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.655s (election complete) I0507 20:08:27.615972 1176 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:27.615988 1178 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:27.617329 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:27.770815 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.583s (SENT) I0507 20:08:27.770946 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:27.771320 1178 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.922s (election complete) I0507 20:08:27.771404 1178 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:27.835345 1178 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:27.835484 1178 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.965s (starting election) I0507 20:08:27.835570 1178 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:27.836109 1178 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:27.836307 1178 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:27.837131 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:27.853637 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.237s (SENT) I0507 20:08:27.853742 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:27.854107 1178 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.480s (election complete) I0507 20:08:27.854189 1178 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:27.952486 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:24.952296 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:24.952629 (+ 333us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:24.952647 (+ 18us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:24.952786 (+ 139us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:27.951279 (+2998493us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":85} W0507 20:08:27.952827 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:28.334612 1178 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:28.334749 1178 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.362s (starting election) I0507 20:08:28.334872 1178 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:28.335337 1178 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:28.335543 1178 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:28.336472 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:28.694253 1178 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:28.694384 1178 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.280s (starting election) I0507 20:08:28.694488 1178 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:28.695098 1178 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:28.695303 1178 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:28.696516 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:28.697628 1178 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:28.697774 1178 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.113s (starting election) I0507 20:08:28.697901 1178 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:28.698539 1178 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:28.698773 1178 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:28.699054 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.362s (SENT) I0507 20:08:28.699138 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:28.699410 1178 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.898s (election complete) I0507 20:08:28.699494 1178 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:28.699647 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:28.801631 1178 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:28.801771 1178 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.536s (starting election) I0507 20:08:28.801880 1178 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:28.802395 1178 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:28.802582 1178 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:28.802934 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.965s (SENT) I0507 20:08:28.803001 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:28.803160 1178 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.378s (election complete) I0507 20:08:28.803243 1178 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:28.803680 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:28.812608 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.113s (SENT) I0507 20:08:28.812702 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:28.813071 1178 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.484s (election complete) I0507 20:08:28.813182 1178 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:28.975749 1178 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:28.975883 1178 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.447s (starting election) I0507 20:08:28.976011 1178 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:28.976493 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.280s (SENT) I0507 20:08:28.976537 1178 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:28.976655 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:28.976778 1178 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:28.977157 1178 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.505s (election complete) I0507 20:08:28.977237 1178 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:28.977895 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:29.181991 1181 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:29.182116 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.238s (starting election) I0507 20:08:29.182204 1181 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:29.182648 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:29.182821 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:29.183565 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:29.298213 1181 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:29.298336 1181 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.336s (starting election) I0507 20:08:29.298442 1181 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:29.298952 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:29.299134 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:29.299985 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:29.339598 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.536s (SENT) I0507 20:08:29.339727 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:29.340201 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.353s (election complete) I0507 20:08:29.340317 1181 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:29.422128 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.238s (SENT) I0507 20:08:29.422242 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:29.422614 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.666s (election complete) I0507 20:08:29.422693 1181 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:29.424919 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.447s (SENT) I0507 20:08:29.425014 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:29.425289 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.138s (election complete) I0507 20:08:29.425356 1181 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:29.564764 1181 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:29.564898 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.108s (starting election) I0507 20:08:29.564993 1181 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:29.565459 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:29.565642 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:29.566460 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:29.635133 1181 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:29.635277 1181 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.291s (starting election) I0507 20:08:29.635390 1181 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:29.636014 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:29.636227 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:29.636333 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.336s (SENT) I0507 20:08:29.636500 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:29.636894 1181 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.822s (election complete) I0507 20:08:29.637009 1181 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:29.637197 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:29.673890 1181 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:29.674017 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.600s (starting election) I0507 20:08:29.674129 1181 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:29.674641 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:29.674751 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.108s (SENT) I0507 20:08:29.674852 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:29.674856 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:29.675191 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.479s (election complete) I0507 20:08:29.675287 1181 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:29.675771 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:29.928761 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.291s (SENT) I0507 20:08:29.928887 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:29.929201 1181 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.264s (election complete) I0507 20:08:29.929318 1181 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:30.090062 1181 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:30.090203 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.418s (starting election) I0507 20:08:30.090306 1181 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:30.090814 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:30.091032 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:30.091946 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:30.155216 1181 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:30.155344 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.413s (starting election) I0507 20:08:30.155439 1181 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:30.155912 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:30.156061 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:30.156867 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:30.194314 1181 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:30.194480 1181 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.430s (starting election) I0507 20:08:30.194622 1181 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:30.195163 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:30.195379 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:30.196342 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:30.276510 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.600s (SENT) I0507 20:08:30.276612 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:30.277000 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.477s (election complete) I0507 20:08:30.277120 1181 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:30.509670 1181 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:30.509730 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.418s (SENT) I0507 20:08:30.509799 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.587s (starting election) I0507 20:08:30.509847 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:30.509912 1181 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:30.510447 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:30.510468 1182 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.764s (election complete) I0507 20:08:30.510536 1182 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:30.510692 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:30.511533 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:30.570176 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.413s (SENT) I0507 20:08:30.570276 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:30.570713 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.270s (election complete) I0507 20:08:30.570838 1181 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:30.625885 1181 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:30.625924 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.430s (SENT) I0507 20:08:30.626051 1181 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.107s (starting election) I0507 20:08:30.626098 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:30.626173 1181 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:30.626754 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:30.626772 1182 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.359s (election complete) I0507 20:08:30.626840 1182 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:30.626976 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:30.627866 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:30.734630 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.107s (SENT) I0507 20:08:30.734748 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:30.735188 1181 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.412s (election complete) I0507 20:08:30.735321 1181 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:30.842209 1181 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:30.842373 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.849s (starting election) I0507 20:08:30.842496 1181 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:30.843097 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:30.843353 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:30.844470 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:31.099458 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.587s (SENT) I0507 20:08:31.099563 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:31.099968 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.178s (election complete) I0507 20:08:31.100106 1181 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:31.148615 1181 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:31.148766 1181 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.943s (starting election) I0507 20:08:31.148871 1181 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:31.149374 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:31.149588 1181 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:31.150568 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:31.279587 1181 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:31.279732 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.107s (starting election) I0507 20:08:31.279841 1181 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:31.280339 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:31.280567 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:31.281536 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:31.387419 1181 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:31.387603 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.468s (starting election) I0507 20:08:31.387706 1181 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:31.388214 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:31.388407 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:31.388869 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.107s (SENT) I0507 20:08:31.388974 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:31.389227 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.467s (election complete) I0507 20:08:31.389322 1181 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:31.389336 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:31.439501 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:28.441301 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:28.441569 (+ 268us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:28.441582 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:28.441679 (+ 97us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:31.439146 (+2997467us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":91} W0507 20:08:31.439704 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:31.692070 1181 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:31.692206 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.127s (starting election) I0507 20:08:31.692337 1181 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:31.692850 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:31.693044 1181 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:31.693373 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.849s (SENT) I0507 20:08:31.693435 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:31.693761 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.299s (election complete) I0507 20:08:31.693874 1181 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:31.693876 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:31.820585 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.127s (SENT) I0507 20:08:31.820677 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:31.820982 1181 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.917s (election complete) I0507 20:08:31.821059 1181 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:31.857117 1181 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:31.857264 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.505s (starting election) I0507 20:08:31.857340 1181 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:31.857619 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.468s (SENT) I0507 20:08:31.857743 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:31.857810 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:31.857964 1181 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:31.858057 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.246s (election complete) I0507 20:08:31.858086 1181 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:31.858647 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:32.092612 1184 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:32.092722 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.638s (starting election) I0507 20:08:32.092798 1184 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:32.093212 1184 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:32.093363 1184 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:32.093714 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.943s (SENT) I0507 20:08:32.093829 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:32.094105 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.230s (election complete) I0507 20:08:32.094207 1184 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:32.094226 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:32.105576 1184 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:32.105680 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.153s (starting election) I0507 20:08:32.105752 1184 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:32.106215 1184 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:32.106364 1184 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:32.107121 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:32.260144 1184 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:32.260246 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.153s (SENT) I0507 20:08:32.260252 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.620s (starting election) I0507 20:08:32.260324 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:32.260376 1184 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:32.260781 1184 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:32.260799 1181 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.200s (election complete) I0507 20:08:32.260849 1181 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:32.260923 1184 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:32.261565 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:32.324829 1184 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:32.324949 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.778s (starting election) I0507 20:08:32.325022 1184 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:32.325469 1184 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:32.325623 1184 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:32.326593 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:32.363730 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.505s (SENT) I0507 20:08:32.363834 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:32.364120 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.869s (election complete) I0507 20:08:32.364218 1184 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:32.732529 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.638s (SENT) I0507 20:08:32.732640 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:32.732976 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.820s (election complete) I0507 20:08:32.733052 1184 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:32.739106 1184 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:32.739225 1184 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.990s (starting election) I0507 20:08:32.739315 1184 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:32.739819 1184 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:32.739996 1184 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:32.740793 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:32.881739 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.620s (SENT) I0507 20:08:32.881866 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:32.882292 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.622s (election complete) I0507 20:08:32.882407 1184 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:33.105589 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.778s (SENT) I0507 20:08:33.105718 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:33.106168 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.281s (election complete) I0507 20:08:33.106266 1184 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:33.388401 1184 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:33.388542 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.692s (starting election) I0507 20:08:33.388638 1184 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:33.389133 1184 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:33.389324 1184 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:33.390604 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:33.505730 1184 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:33.505854 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.479s (starting election) I0507 20:08:33.505937 1184 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:33.506443 1184 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:33.506618 1184 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:33.507433 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:33.729979 1184 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:33.730109 1184 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.396s (starting election) I0507 20:08:33.730190 1184 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:33.730681 1184 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:33.730849 1184 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:33.730868 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.990s (SENT) I0507 20:08:33.730989 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:33.731259 1184 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.541s (election complete) I0507 20:08:33.731333 1184 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:33.731704 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:33.986099 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.479s (SENT) I0507 20:08:33.986193 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:33.986202 1184 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:33.986299 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.124s (starting election) I0507 20:08:33.986382 1184 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:33.986871 1184 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:33.987080 1184 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:33.987179 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.082s (election complete) I0507 20:08:33.987220 1184 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:33.987865 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:34.070165 1187 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:34.070298 1187 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.165s (starting election) I0507 20:08:34.070391 1187 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:34.071018 1187 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:34.071207 1187 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:34.071871 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:34.081063 1187 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:34.081141 1187 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.601s (starting election) I0507 20:08:34.081207 1187 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:34.081621 1187 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:34.081764 1187 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:34.082026 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.692s (SENT) I0507 20:08:34.082096 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:34.082388 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.227s (election complete) W0507 20:08:34.082448 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:34.082484 1184 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:34.111703 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.124s (SENT) I0507 20:08:34.111842 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:34.112185 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.402s (election complete) I0507 20:08:34.112298 1184 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:34.127763 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.396s (SENT) I0507 20:08:34.127882 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:34.128288 1184 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.543s (election complete) I0507 20:08:34.128399 1184 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:34.237335 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.165s (SENT) I0507 20:08:34.237481 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:34.237908 1184 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.932s (election complete) I0507 20:08:34.238029 1184 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:34.310220 1184 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:34.310338 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.920s (starting election) I0507 20:08:34.310415 1184 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:34.310914 1184 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:34.311059 1184 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:34.311923 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:34.672600 1184 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:34.672722 1184 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.115s (starting election) I0507 20:08:34.672807 1184 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:34.673266 1184 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:34.673429 1184 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:34.674270 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:34.684448 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.601s (SENT) I0507 20:08:34.684562 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:34.684923 1184 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.875s (election complete) I0507 20:08:34.685047 1184 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:34.788650 1184 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:34.788770 1184 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.412s (starting election) I0507 20:08:34.788871 1184 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:34.789494 1184 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:34.789501 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.115s (SENT) I0507 20:08:34.789561 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:34.789686 1184 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:34.789849 1184 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.761s (election complete) I0507 20:08:34.789904 1184 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:34.790431 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:34.931481 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:31.933841 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:31.934112 (+ 271us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:31.934126 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:31.934236 (+ 110us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:34.931203 (+2996967us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":82} W0507 20:08:34.931748 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:35.171146 1188 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:35.171340 1188 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.888s (starting election) I0507 20:08:35.171463 1188 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:35.172057 1188 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:35.172243 1188 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:35.173210 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:35.203486 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.412s (SENT) I0507 20:08:35.203577 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:35.203866 1188 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.574s (election complete) I0507 20:08:35.203958 1188 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:35.231650 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.920s (SENT) I0507 20:08:35.231743 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:35.232091 1188 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.854s (election complete) I0507 20:08:35.232178 1188 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:35.779353 1191 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:35.779449 1191 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.140s (starting election) I0507 20:08:35.779523 1191 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:35.779958 1191 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:35.780117 1191 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:35.781513 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:35.920742 1191 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:35.920794 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.140s (SENT) I0507 20:08:35.920876 1191 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.647s (starting election) I0507 20:08:35.920903 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:35.920959 1191 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:35.922560 1191 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:35.922610 1193 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.989s (election complete) I0507 20:08:35.922662 1193 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:35.922797 1191 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:35.923722 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:36.060458 1191 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:36.060582 1191 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.825s (starting election) I0507 20:08:36.060680 1191 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:36.061139 1191 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:36.061266 1191 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:36.061578 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.888s (SENT) I0507 20:08:36.061736 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:08:36.062017 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:36.062033 1191 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.848s (election complete) I0507 20:08:36.062098 1191 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:36.087177 1191 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:36.087291 1191 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.836s (starting election) I0507 20:08:36.087371 1191 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:36.087942 1191 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:36.088078 1191 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:36.089057 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:36.570516 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.647s (SENT) I0507 20:08:36.570617 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:36.570909 1191 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.614s (election complete) I0507 20:08:36.570973 1191 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:36.887362 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.825s (SENT) I0507 20:08:36.887506 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:36.887970 1191 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.331s (election complete) I0507 20:08:36.888063 1191 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:36.924731 1191 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:36.924845 1191 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.792s (starting election) W0507 20:08:36.924835 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.836s (SENT) I0507 20:08:36.924958 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:36.924919 1191 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:36.925334 1191 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:36.925506 1191 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:36.925534 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.129s (election complete) I0507 20:08:36.925611 1194 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:36.926709 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:37.055415 1194 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:37.055527 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.574s (starting election) I0507 20:08:37.055603 1194 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:37.056087 1194 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:37.056234 1194 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:37.057232 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:37.185477 1194 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:37.185664 1194 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.501s (starting election) I0507 20:08:37.185783 1194 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:37.186429 1194 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:37.186687 1194 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:37.187597 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:37.220439 1194 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:37.220577 1194 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.806s (starting election) I0507 20:08:37.220674 1194 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:37.221294 1194 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:37.221510 1194 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:37.222632 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:37.630210 1194 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:37.630349 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.156s (starting election) I0507 20:08:37.630439 1194 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:37.630929 1194 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:37.631109 1194 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:37.631290 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.574s (SENT) I0507 20:08:37.631382 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:37.631562 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.639s (election complete) I0507 20:08:37.631626 1194 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:37.632211 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:37.688042 1194 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:37.688174 1194 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.247s (starting election) I0507 20:08:37.688287 1194 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:37.688757 1194 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:37.688954 1194 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:37.689224 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.501s (SENT) I0507 20:08:37.689282 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:37.689544 1194 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.921s (election complete) I0507 20:08:37.689627 1194 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:37.690085 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:37.719461 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.792s (SENT) I0507 20:08:37.719570 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:37.719875 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.655s (election complete) I0507 20:08:37.719966 1194 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:37.787422 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.156s (SENT) I0507 20:08:37.787551 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:37.787899 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.426s (election complete) I0507 20:08:37.787995 1194 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:37.937307 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.247s (SENT) I0507 20:08:37.937412 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:37.937772 1194 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.837s (election complete) I0507 20:08:37.937860 1194 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:38.027673 1194 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:38.027820 1194 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.422s (starting election) I0507 20:08:38.027911 1194 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:38.028398 1194 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:38.028570 1194 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:38.028832 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.806s (SENT) I0507 20:08:38.028890 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:38.029117 1194 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.223s (election complete) I0507 20:08:38.029204 1194 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:38.029498 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:38.215025 1194 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:38.215173 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.500s (starting election) I0507 20:08:38.215302 1194 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:38.215914 1194 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:38.216138 1194 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:38.217012 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:38.253284 1194 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:38.253414 1194 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.635s (starting election) I0507 20:08:38.253526 1194 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:38.254099 1194 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:38.254320 1194 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:38.255208 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:38.439556 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:35.440620 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:35.441028 (+ 408us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:35.441050 (+ 22us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:35.441238 (+ 188us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:38.439205 (+2997967us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":137} W0507 20:08:38.439819 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:08:38.451108 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.422s (SENT) I0507 20:08:38.451228 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:38.451568 1194 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.337s (election complete) I0507 20:08:38.451689 1194 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:38.715915 1194 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:38.716032 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.629s (starting election) I0507 20:08:38.716122 1194 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:38.716614 1194 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:38.716811 1194 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:38.717007 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.500s (SENT) I0507 20:08:38.717083 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:38.717381 1194 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.719s (election complete) I0507 20:08:38.717528 1194 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:38.717578 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:38.775243 1194 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:38.775393 1194 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.400s (starting election) I0507 20:08:38.775494 1194 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:38.776011 1194 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:38.776201 1194 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:38.777083 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:38.789609 1194 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:38.789749 1194 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.165s (starting election) I0507 20:08:38.789896 1194 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:38.790426 1194 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:38.790624 1194 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:38.791546 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:38.891281 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.635s (SENT) I0507 20:08:38.891407 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:38.891793 1194 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.644s (election complete) I0507 20:08:38.891904 1194 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:38.956537 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.165s (SENT) I0507 20:08:38.956665 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:38.957090 1194 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.872s (election complete) I0507 20:08:38.957201 1194 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:39.176359 1194 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:39.176498 1194 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.161s (starting election) I0507 20:08:39.176631 1194 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:39.176769 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.400s (SENT) I0507 20:08:39.176878 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:39.177340 1194 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:39.177577 1194 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:39.177575 1198 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.780s (election complete) I0507 20:08:39.177691 1198 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:39.178298 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:39.338937 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.161s (SENT) I0507 20:08:39.339041 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:39.339356 1198 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.561s (election complete) I0507 20:08:39.339445 1198 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:39.347673 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.629s (SENT) I0507 20:08:39.347781 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:39.348067 1198 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.719s (election complete) I0507 20:08:39.348158 1198 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:39.830013 1198 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:39.830160 1198 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.902s (starting election) I0507 20:08:39.830281 1198 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:39.830791 1198 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:39.831050 1198 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:39.832233 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:39.900832 1198 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:39.900969 1198 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.373s (starting election) I0507 20:08:39.901072 1198 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:39.901553 1198 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:39.901736 1198 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:39.902436 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:40.067736 1198 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:40.068573 1198 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.203s (starting election) I0507 20:08:40.068692 1198 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:40.069208 1198 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:40.069373 1198 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:40.070230 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:40.272308 1198 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:40.272455 1198 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.609s (starting election) I0507 20:08:40.272567 1198 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:40.273120 1198 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:40.273320 1198 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:40.273447 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.203s (SENT) I0507 20:08:40.273579 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:40.273851 1198 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.233s (election complete) I0507 20:08:40.273918 1198 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:40.274170 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:40.275032 1198 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:40.275084 1198 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.932s (starting election) I0507 20:08:40.275146 1198 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:40.275466 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.373s (SENT) I0507 20:08:40.275545 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:40.275559 1198 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:40.275705 1198 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:40.275825 1198 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.921s (election complete) I0507 20:08:40.275918 1198 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:40.276338 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:40.508253 1200 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:40.508385 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.267s (starting election) I0507 20:08:40.508488 1200 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:40.508976 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:40.509191 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:40.510165 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:40.733224 1200 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:40.733351 1200 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.829s (starting election) I0507 20:08:40.733510 1200 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:40.733981 1200 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:40.733952 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.902s (SENT) I0507 20:08:40.734071 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:40.734150 1200 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:40.734369 1200 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.245s (election complete) I0507 20:08:40.734421 1200 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:40.735146 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:40.776891 1200 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:40.777016 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.814s (starting election) W0507 20:08:40.777017 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.267s (SENT) I0507 20:08:40.777132 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:40.777106 1200 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:40.777578 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:40.777619 1201 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.310s (election complete) I0507 20:08:40.777706 1201 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:40.777797 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:40.778990 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:40.883780 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.609s (SENT) I0507 20:08:40.883885 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:40.884246 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.359s (election complete) I0507 20:08:40.884337 1200 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:40.980298 1200 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:40.980456 1200 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.782s (starting election) I0507 20:08:40.980566 1200 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:40.981119 1200 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:40.981325 1200 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:40.982277 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:41.197912 1200 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:41.198053 1200 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.844s (starting election) I0507 20:08:41.198159 1200 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:41.198700 1200 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:41.198940 1200 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:41.199965 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:41.208714 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.932s (SENT) I0507 20:08:41.208804 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:41.209115 1200 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.342s (election complete) I0507 20:08:41.209236 1200 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:41.244729 1200 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:41.244884 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.376s (starting election) I0507 20:08:41.244999 1200 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:41.245525 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:41.245725 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:41.246778 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:41.552255 1200 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:41.552366 1200 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.839s (starting election) I0507 20:08:41.552487 1200 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:41.552949 1200 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:41.553125 1200 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:41.554121 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:41.564941 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.829s (SENT) I0507 20:08:41.565052 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:41.565414 1200 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.915s (election complete) I0507 20:08:41.565534 1200 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:41.593466 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.814s (SENT) I0507 20:08:41.593578 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:41.594005 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.811s (election complete) I0507 20:08:41.594128 1200 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:41.622685 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.376s (SENT) I0507 20:08:41.622797 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:41.623203 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.218s (election complete) I0507 20:08:41.623322 1200 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:41.764659 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.782s (SENT) I0507 20:08:41.764775 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:41.765100 1200 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.755s (election complete) I0507 20:08:41.765228 1200 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:41.841751 1200 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:41.841898 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.748s (starting election) I0507 20:08:41.842007 1200 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:41.842514 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:41.842716 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:41.843767 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:41.959600 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:38.960729 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:38.960966 (+ 237us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:38.960980 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:38.961083 (+ 103us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:41.959250 (+2998167us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":73} W0507 20:08:41.960714 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:08:42.045984 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.844s (SENT) I0507 20:08:42.046093 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:42.046504 1200 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.540s (election complete) I0507 20:08:42.046615 1200 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:42.393949 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.839s (SENT) I0507 20:08:42.394078 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:42.394546 1200 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.219s (election complete) I0507 20:08:42.394667 1200 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:42.520869 1200 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:42.520983 1200 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.870s (starting election) I0507 20:08:42.521087 1200 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:42.521548 1200 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:42.521734 1200 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:42.522716 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:42.590914 1200 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:42.591034 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.934s (starting election) I0507 20:08:42.591133 1200 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:42.591598 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:42.592057 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.748s (SENT) I0507 20:08:42.592133 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:42.592697 1205 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.505s (election complete) I0507 20:08:42.592746 1205 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:42.601985 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:42.602802 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:42.614504 1200 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:42.614606 1200 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.978s (starting election) I0507 20:08:42.614697 1200 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:42.615133 1200 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:42.615320 1200 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:42.616298 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:43.099068 1200 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:43.099162 1200 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.439s (starting election) I0507 20:08:43.099228 1200 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:43.099648 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:43.099776 1200 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:43.100853 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:43.392001 1200 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:43.392172 1200 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.202s (starting election) I0507 20:08:43.392314 1200 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:43.392948 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.870s (SENT) I0507 20:08:43.392984 1200 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:43.393030 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:43.393229 1200 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:43.393404 1200 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.747s (election complete) I0507 20:08:43.393472 1200 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:43.393968 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:43.537200 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.934s (SENT) I0507 20:08:43.537289 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:43.537556 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.672s (election complete) I0507 20:08:43.537627 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:43.539840 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.439s (SENT) I0507 20:08:43.539899 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:43.540071 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.243s (election complete) I0507 20:08:43.540122 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:43.593420 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:43.593540 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.768s (starting election) I0507 20:08:43.593612 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:43.594022 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:43.594216 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:43.594475 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.978s (SENT) I0507 20:08:43.594532 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:43.594688 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.469s (election complete) I0507 20:08:43.594755 1206 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:43.595310 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:43.596071 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.202s (SENT) I0507 20:08:43.596114 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:43.596300 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.326s (election complete) I0507 20:08:43.596361 1206 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:43.783838 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:43.783946 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.954s (starting election) I0507 20:08:43.784018 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:43.784448 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:43.784598 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:43.785328 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:43.923279 1206 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:43.923449 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.344s (starting election) I0507 20:08:43.923593 1206 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:43.924293 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:43.924540 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:43.925375 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:44.065028 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:44.065145 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.746s (starting election) I0507 20:08:44.065223 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:44.065706 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:44.065963 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:44.066998 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:44.267978 1206 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:44.268132 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.959s (starting election) I0507 20:08:44.268221 1206 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:44.268725 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:44.268908 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:44.269196 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.344s (SENT) I0507 20:08:44.269250 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:44.269412 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.755s (election complete) I0507 20:08:44.269469 1206 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:44.269726 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:44.363812 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.768s (SENT) I0507 20:08:44.363929 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:44.364251 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.288s (election complete) I0507 20:08:44.364351 1206 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:44.653808 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:44.653934 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.778s (starting election) I0507 20:08:44.654023 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:44.654500 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:44.654669 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:44.655479 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:44.738956 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:44.739073 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.423s (starting election) I0507 20:08:44.739151 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:44.739611 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:44.739784 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:44.739995 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.954s (SENT) I0507 20:08:44.740057 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:44.740281 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.372s (election complete) I0507 20:08:44.740334 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:44.740783 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:44.813012 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.746s (SENT) I0507 20:08:44.813107 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:44.813398 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.319s (election complete) I0507 20:08:44.813468 1206 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:45.025168 1206 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:45.025297 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.944s (starting election) I0507 20:08:45.025375 1206 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:45.025825 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:45.025996 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:45.026921 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:45.112993 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:45.113109 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.858s (starting election) I0507 20:08:45.113191 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:45.113643 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:45.113840 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:45.114812 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:45.133764 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:45.133882 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.893s (starting election) I0507 20:08:45.133958 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:45.134407 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:45.134577 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:45.135368 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:45.163445 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.423s (SENT) I0507 20:08:45.163561 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:45.163931 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.117s (election complete) I0507 20:08:45.164031 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:45.229552 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.959s (SENT) I0507 20:08:45.229671 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:45.229976 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.737s (election complete) I0507 20:08:45.230062 1206 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:45.282390 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:45.282491 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.539s (starting election) I0507 20:08:45.282560 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:45.282974 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:45.283099 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:45.283884 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:45.434176 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.778s (SENT) I0507 20:08:45.434294 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:45.434655 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.733s (election complete) I0507 20:08:45.434773 1206 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:45.479490 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:42.479526 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:42.479806 (+ 280us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:42.479821 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:42.479915 (+ 94us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:45.479171 (+2999256us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":74} W0507 20:08:45.479725 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:45.822044 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:45.822229 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.186s (starting election) I0507 20:08:45.822389 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:45.822984 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:45.823242 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:45.823221 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.539s (SENT) I0507 20:08:45.823364 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:45.823632 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.075s (election complete) I0507 20:08:45.823689 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:45.824368 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:45.899181 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:45.899282 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.756s (starting election) I0507 20:08:45.899350 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:45.899781 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:45.899920 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:45.900633 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:45.967788 1206 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:45.967907 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.749s (starting election) I0507 20:08:45.968001 1206 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:45.968483 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:45.968658 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:45.969530 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:45.971379 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.944s (SENT) I0507 20:08:45.971442 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:45.971658 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.948s (election complete) I0507 20:08:45.971712 1206 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:45.972812 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.858s (SENT) I0507 20:08:45.972856 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:45.972966 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.342s (election complete) I0507 20:08:45.972995 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:46.010324 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.186s (SENT) I0507 20:08:46.010429 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:46.010787 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.922s (election complete) I0507 20:08:46.010951 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:46.029222 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.893s (SENT) I0507 20:08:46.029330 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:46.029688 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.331s (election complete) I0507 20:08:46.029781 1206 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:46.361788 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:46.361915 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.380s (starting election) I0507 20:08:46.362040 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:46.362576 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:46.362768 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:46.363700 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:46.657759 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.756s (SENT) I0507 20:08:46.657857 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:46.658201 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.273s (election complete) I0507 20:08:46.658300 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:46.718751 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.749s (SENT) I0507 20:08:46.718868 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:46.719295 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.390s (election complete) I0507 20:08:46.719393 1206 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:46.742759 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:46.742929 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.424s (starting election) I0507 20:08:46.743021 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:46.743504 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:46.743691 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:46.743780 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.380s (SENT) I0507 20:08:46.744076 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:46.744323 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.143s (election complete) I0507 20:08:46.744371 1206 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:46.744447 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:46.888159 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:46.888295 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.645s (starting election) I0507 20:08:46.888383 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:46.888901 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:46.889099 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:46.889976 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:46.932144 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:46.932260 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.148s (starting election) I0507 20:08:46.932341 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:46.932785 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:46.932957 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:46.933755 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:47.081118 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:08:47.081194 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.148s (SENT) I0507 20:08:47.081213 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.281s (starting election) I0507 20:08:47.081323 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:47.081849 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:47.082051 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:47.082621 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:47.082850 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.517s (election complete) I0507 20:08:47.082943 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:47.082988 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:47.110043 1206 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:47.110183 1206 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.838s (starting election) I0507 20:08:47.110280 1206 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:47.110767 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:47.110960 1206 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:47.111768 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:47.168997 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.424s (SENT) I0507 20:08:47.169096 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:47.169369 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.710s (election complete) I0507 20:08:47.169438 1206 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:47.364717 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.281s (SENT) I0507 20:08:47.364821 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:47.365154 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.541s (election complete) I0507 20:08:47.365238 1206 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:47.535562 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.645s (SENT) I0507 20:08:47.535678 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:47.536036 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.102s (election complete) I0507 20:08:47.536118 1206 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:47.638824 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:47.638955 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.308s (starting election) I0507 20:08:47.639040 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:47.639538 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:47.639709 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:47.640671 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:47.906857 1206 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:47.906993 1206 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.522s (starting election) I0507 20:08:47.907071 1206 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:47.907558 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:47.907727 1206 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:47.908561 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:47.947803 1206 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:47.947926 1206 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.886s (starting election) W0507 20:08:47.948022 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.308s (SENT) I0507 20:08:47.948098 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:47.948019 1206 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:47.948622 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:47.948640 1211 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.526s (election complete) I0507 20:08:47.948725 1211 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:47.948792 1206 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:47.948799 1211 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:47.948849 1211 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.237s (starting election) I0507 20:08:47.948905 1211 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:47.949518 1211 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:47.949726 1211 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:47.949862 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:47.949925 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:47.950991 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.838s (SENT) I0507 20:08:47.951040 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:47.951215 1212 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.930s (election complete) I0507 20:08:47.951277 1212 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:48.188037 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.237s (SENT) I0507 20:08:48.188136 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:48.188463 1212 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.241s (election complete) I0507 20:08:48.188550 1212 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:48.429574 1212 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:48.429675 1212 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.410s (starting election) I0507 20:08:48.429767 1211 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:48.429868 1211 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.677s (starting election) I0507 20:08:48.429795 1212 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:48.429957 1211 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:48.430296 1212 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:48.430405 1211 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:48.430459 1212 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:48.430544 1211 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:48.430863 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.522s (SENT) I0507 20:08:48.431011 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:48.431305 1211 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.799s (election complete) I0507 20:08:48.431375 1211 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:48.431550 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:48.431646 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:48.475191 1211 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:48.475304 1211 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.648s (starting election) I0507 20:08:48.475397 1211 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:48.475841 1211 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:48.476059 1211 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:48.477005 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:48.838466 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.886s (SENT) I0507 20:08:48.838572 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:48.838891 1211 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.550s (election complete) I0507 20:08:48.838987 1211 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:48.841212 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.410s (SENT) I0507 20:08:48.841315 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:48.841622 1211 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.741s (election complete) I0507 20:08:48.841758 1211 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:49.011544 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:46.014011 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:46.014294 (+ 283us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:46.014309 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:46.014412 (+ 103us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:49.011221 (+2996809us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":89} W0507 20:08:49.011852 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:49.107939 1211 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:49.108098 1211 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (starting election) I0507 20:08:49.108222 1211 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:49.108633 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.677s (SENT) I0507 20:08:49.108721 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:49.108862 1211 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:49.109118 1211 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:49.109272 1213 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.726s (election complete) I0507 20:08:49.109375 1213 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:49.110458 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:49.125272 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.648s (SENT) I0507 20:08:49.125383 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:49.125695 1213 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.420s (election complete) I0507 20:08:49.125777 1213 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:49.209755 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.099s (SENT) I0507 20:08:49.209888 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:49.210245 1213 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.156s (election complete) I0507 20:08:49.210358 1213 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:49.367621 1213 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:49.367763 1213 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.590s (starting election) I0507 20:08:49.367859 1213 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:49.368355 1213 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:49.368549 1213 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:49.369545 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:49.546825 1213 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:49.546944 1213 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.094s (starting election) I0507 20:08:49.547030 1213 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:49.547428 1213 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:49.547577 1213 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:49.548351 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:49.583310 1213 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:49.583421 1213 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.520s (starting election) I0507 20:08:49.583510 1213 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:49.583963 1213 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:49.584139 1213 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:49.585175 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:49.641822 1213 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:49.641957 1213 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.864s (starting election) I0507 20:08:49.642108 1213 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:49.642535 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.094s (SENT) I0507 20:08:49.642601 1213 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:49.642648 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:49.642791 1213 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:49.643038 1213 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.640s (election complete) I0507 20:08:49.643119 1213 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:49.643647 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:49.959033 1217 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:49.959167 1217 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.691s (starting election) I0507 20:08:49.959316 1217 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:49.959789 1217 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:49.959998 1217 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:49.960525 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.590s (SENT) I0507 20:08:49.960670 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:08:49.960844 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:49.961457 1217 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.147s (election complete) I0507 20:08:49.961557 1217 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:50.104197 1217 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:50.104333 1217 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.370s (starting election) I0507 20:08:50.104425 1217 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:50.104871 1217 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:50.105067 1217 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:50.105499 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.520s (SENT) I0507 20:08:50.105638 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:08:50.105963 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:50.105983 1217 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.795s (election complete) I0507 20:08:50.106062 1217 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:50.109532 1217 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:50.109653 1217 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.399s (starting election) I0507 20:08:50.109756 1217 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:50.110188 1217 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:50.110339 1217 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:50.111315 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:50.283522 1217 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:50.283639 1217 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.215s (starting election) I0507 20:08:50.283728 1217 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:50.284160 1217 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:50.284320 1217 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:50.285326 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:50.475481 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.370s (SENT) I0507 20:08:50.475642 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:50.476043 1217 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (election complete) I0507 20:08:50.476145 1217 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:50.499680 1217 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:50.499817 1217 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.150s (starting election) W0507 20:08:50.499868 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.215s (SENT) I0507 20:08:50.500025 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:50.499974 1217 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:50.500573 1217 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:50.500733 1218 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.945s (election complete) I0507 20:08:50.500782 1217 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:50.500834 1218 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:50.501436 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:50.508811 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.864s (SENT) I0507 20:08:50.508952 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:50.509373 1218 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.731s (election complete) I0507 20:08:50.509526 1218 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:50.509649 1217 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:50.509776 1217 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.333s (starting election) I0507 20:08:50.509892 1217 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:50.510572 1217 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:50.510601 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.399s (SENT) I0507 20:08:50.510673 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:50.510848 1217 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:50.511106 1217 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.221s (election complete) I0507 20:08:50.511186 1217 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:50.511627 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:50.651446 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.150s (SENT) I0507 20:08:50.651607 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:50.652034 1218 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.769s (election complete) I0507 20:08:50.652159 1218 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:50.653407 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.691s (SENT) I0507 20:08:50.653527 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:50.653884 1218 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.096s (election complete) I0507 20:08:50.654012 1218 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:50.751500 1218 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:50.751665 1218 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.938s (starting election) I0507 20:08:50.751809 1218 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:50.752470 1218 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:50.752737 1218 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:50.753808 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:50.844482 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.333s (SENT) I0507 20:08:50.844602 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:50.844969 1218 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.268s (election complete) I0507 20:08:50.845099 1218 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:50.926522 1218 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:50.926667 1218 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.193s (starting election) I0507 20:08:50.926789 1218 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:50.927448 1218 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:50.927701 1218 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:50.928664 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:51.113956 1218 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:51.114085 1218 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.550s (starting election) I0507 20:08:51.114177 1218 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:51.114647 1218 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:51.114837 1218 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:51.115798 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:51.120338 1218 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:51.120419 1218 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.425s (starting election) I0507 20:08:51.120494 1218 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:51.120944 1218 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:51.121074 1218 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:51.121289 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.193s (SENT) I0507 20:08:51.121372 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:51.121604 1218 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.589s (election complete) I0507 20:08:51.121708 1218 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:51.121994 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:51.421669 1218 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:51.421798 1218 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.834s (starting election) I0507 20:08:51.421910 1218 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:51.422358 1218 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:51.422518 1218 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:51.423413 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:51.546898 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.425s (SENT) I0507 20:08:51.547029 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:51.547454 1218 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.572s (election complete) I0507 20:08:51.547564 1218 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:51.665416 1218 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:51.665573 1218 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.746s (starting election) I0507 20:08:51.665702 1218 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:51.666064 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.550s (SENT) I0507 20:08:51.666198 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:51.666497 1218 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:51.666847 1218 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:51.667666 1218 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.457s (election complete) I0507 20:08:51.667749 1218 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:51.667884 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:51.691828 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.938s (SENT) I0507 20:08:51.691957 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:51.692333 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.348s (election complete) I0507 20:08:51.692453 1219 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:52.041823 1219 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:52.041949 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.732s (starting election) I0507 20:08:52.042062 1219 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:52.042536 1219 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:52.042707 1219 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:52.043599 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:52.120427 1219 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:52.120568 1219 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.426s (starting election) I0507 20:08:52.120676 1219 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:52.121217 1219 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:52.121438 1219 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:52.122354 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:52.256952 1219 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:52.257088 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.323s (starting election) I0507 20:08:52.257199 1219 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:52.257709 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:52.257874 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:52.258316 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.834s (SENT) I0507 20:08:52.258383 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:08:52.258546 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:52.258653 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.130s (election complete) I0507 20:08:52.258752 1219 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:52.390091 1219 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:52.390208 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.245s (starting election) I0507 20:08:52.390297 1219 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:52.390741 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:52.390959 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:52.391695 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:52.415642 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.746s (SENT) I0507 20:08:52.415777 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:52.416172 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.410s (election complete) I0507 20:08:52.416260 1219 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:52.547070 1219 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:52.547184 1219 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.317s (starting election) I0507 20:08:52.547281 1219 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:52.547749 1219 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:52.547919 1219 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:52.548205 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.426s (SENT) I0507 20:08:52.548313 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:52.548570 1219 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.878s (election complete) I0507 20:08:52.548648 1219 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:52.548872 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:52.551399 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:49.551924 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:49.552153 (+ 229us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:49.552167 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:49.552266 (+ 99us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:52.551119 (+2998853us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":73} W0507 20:08:52.551740 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:08:52.582186 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.323s (SENT) I0507 20:08:52.582298 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:52.582618 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.877s (election complete) I0507 20:08:52.582732 1219 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:52.637076 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.245s (SENT) I0507 20:08:52.637195 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:52.637493 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.591s (election complete) I0507 20:08:52.637584 1219 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:52.775413 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.732s (SENT) I0507 20:08:52.775528 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:52.775830 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.104s (election complete) I0507 20:08:52.775923 1219 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:52.865799 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.317s (SENT) I0507 20:08:52.865923 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:52.866261 1219 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.741s (election complete) I0507 20:08:52.866355 1219 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:52.881207 1219 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:52.881314 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.697s (starting election) I0507 20:08:52.881392 1219 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:52.881834 1219 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:52.881989 1219 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:52.883121 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:53.229245 1219 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:53.229378 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.257s (starting election) I0507 20:08:53.229478 1219 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:53.229926 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:53.230092 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:53.231071 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:53.486845 1219 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:53.486982 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.255s (starting election) I0507 20:08:53.487078 1219 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:53.487540 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:53.487736 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:53.487846 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.257s (SENT) I0507 20:08:53.487985 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:53.488286 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.587s (election complete) I0507 20:08:53.488359 1219 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:53.488517 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:53.579198 1219 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:53.579319 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.228s (starting election) I0507 20:08:53.579403 1219 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:53.579869 1219 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:53.580055 1219 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:53.580518 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.697s (SENT) I0507 20:08:53.580662 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:08:53.580822 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:53.580934 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.835s (election complete) I0507 20:08:53.580996 1219 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:53.608518 1219 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:53.608662 1219 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.168s (starting election) I0507 20:08:53.614575 1219 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:53.615136 1219 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:53.615335 1219 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:53.616199 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:53.743775 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.255s (SENT) I0507 20:08:53.743882 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:53.744212 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.508s (election complete) I0507 20:08:53.744307 1219 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:53.783560 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.168s (SENT) I0507 20:08:53.783538 1219 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:53.783656 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:53.783694 1219 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.190s (starting election) I0507 20:08:53.783888 1219 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:53.784615 1219 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:53.784617 1222 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.985s (election complete) I0507 20:08:53.784730 1222 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:53.784837 1219 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:53.785698 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:53.808676 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.228s (SENT) I0507 20:08:53.808825 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:53.809300 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.411s (election complete) I0507 20:08:53.809432 1219 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:53.975909 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.190s (SENT) I0507 20:08:53.976030 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:53.976444 1219 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.931s (election complete) I0507 20:08:53.976574 1219 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:54.221179 1219 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:54.221303 1219 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.583s (starting election) I0507 20:08:54.221396 1219 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:54.221855 1219 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:54.222038 1219 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:54.223001 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:54.253394 1219 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:54.253496 1219 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.702s (starting election) I0507 20:08:54.253578 1219 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:54.253978 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:54.254109 1219 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:54.254729 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:54.805934 1223 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:54.806040 1223 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.521s (starting election) I0507 20:08:54.806131 1223 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:54.806555 1223 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:54.806740 1223 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:54.806903 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.583s (SENT) I0507 20:08:54.806978 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:54.807176 1223 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.589s (election complete) I0507 20:08:54.807258 1223 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:54.808038 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:54.908681 1223 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:54.908812 1223 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.488s (starting election) I0507 20:08:54.908902 1223 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:54.909476 1223 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:54.909734 1223 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:54.910629 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:54.956198 1223 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:54.956326 1223 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.729s (starting election) I0507 20:08:54.956423 1223 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:54.956822 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.702s (SENT) I0507 20:08:54.956892 1223 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:54.956914 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:54.957080 1223 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:54.957262 1223 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.419s (election complete) I0507 20:08:54.957339 1223 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:54.958012 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:55.330509 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.521s (SENT) I0507 20:08:55.330631 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:55.330965 1225 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.209s (election complete) I0507 20:08:55.331043 1225 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:55.376904 1225 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:55.377022 1225 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.998s (starting election) I0507 20:08:55.377122 1225 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:55.377604 1225 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:55.377801 1225 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:55.378763 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:55.397222 1225 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:55.397339 1225 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.815s (starting election) I0507 20:08:55.397433 1225 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:55.397878 1225 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:55.398052 1225 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:55.398319 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.488s (SENT) I0507 20:08:55.398409 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:55.398633 1225 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.470s (election complete) I0507 20:08:55.398694 1225 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:55.399204 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:55.541148 1225 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:55.541272 1225 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.222s (starting election) I0507 20:08:55.541376 1225 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:55.541851 1225 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:55.542040 1225 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:55.543056 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:55.687836 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.729s (SENT) I0507 20:08:55.687969 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:55.688371 1225 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.229s (election complete) I0507 20:08:55.688484 1225 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:55.763921 1225 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:55.764045 1225 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.958s (starting election) I0507 20:08:55.764127 1225 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:55.764492 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.222s (SENT) I0507 20:08:55.764575 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:55.764603 1225 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:55.764775 1225 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:55.764894 1225 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.874s (election complete) I0507 20:08:55.764946 1225 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:55.765691 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:55.869869 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:55.869995 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.691s (starting election) I0507 20:08:55.870113 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:55.870550 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:55.870731 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:55.871726 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:55.918112 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:55.918223 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.638s (starting election) I0507 20:08:55.918305 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:55.918732 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:55.918922 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:55.919517 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:56.103518 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:53.102785 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:53.102994 (+ 209us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:53.103006 (+ 12us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:53.103091 (+ 85us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:56.103204 (+3000113us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":57} W0507 20:08:56.103868 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:08:56.214809 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.815s (SENT) I0507 20:08:56.214947 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:56.215275 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.847s (election complete) I0507 20:08:56.215374 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:56.378106 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.998s (SENT) I0507 20:08:56.378214 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:56.378513 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.313s (election complete) I0507 20:08:56.378602 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:56.558488 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.638s (SENT) I0507 20:08:56.558604 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:56.559022 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.861s (election complete) I0507 20:08:56.559124 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:56.563302 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.691s (SENT) I0507 20:08:56.563410 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:56.563735 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.383s (election complete) I0507 20:08:56.563817 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:56.639302 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:56.639505 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.560s (starting election) I0507 20:08:56.639605 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:56.640072 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:56.640233 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:56.641240 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:56.724215 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.958s (SENT) I0507 20:08:56.724316 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:56.724695 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.148s (election complete) I0507 20:08:56.724822 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:56.873205 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:56.873328 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (starting election) I0507 20:08:56.873430 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:56.873889 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:56.874080 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:56.875149 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:56.947201 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:56.947324 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.769s (starting election) I0507 20:08:56.947412 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:56.947816 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:56.947988 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:56.948732 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:57.201455 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.560s (SENT) I0507 20:08:57.201556 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:57.201839 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.641s (election complete) I0507 20:08:57.201993 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:57.323842 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.449s (SENT) I0507 20:08:57.323949 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:57.324270 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.251s (election complete) I0507 20:08:57.324367 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:08:57.421242 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:57.421355 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.139s (starting election) I0507 20:08:57.421438 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:57.421901 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:57.422062 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:57.423102 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:57.561487 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.139s (SENT) I0507 20:08:57.561594 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:57.561704 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:57.561794 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.162s (starting election) I0507 20:08:57.561872 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:57.562315 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:57.562324 1230 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.442s (election complete) I0507 20:08:57.562439 1230 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:57.562491 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:57.563277 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:57.576263 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:57.576390 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.888s (starting election) I0507 20:08:57.576503 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:57.576936 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:57.577117 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:57.578044 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:57.717236 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:57.717372 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.347s (starting election) I0507 20:08:57.717473 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:57.717902 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:57.718096 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:57.718345 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.769s (SENT) I0507 20:08:57.718417 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:57.718621 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.332s (election complete) I0507 20:08:57.718680 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:57.719275 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:57.725193 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.162s (SENT) I0507 20:08:57.725277 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:57.725549 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.548s (election complete) I0507 20:08:57.725633 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:58.052095 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:58.052215 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.552s (starting election) I0507 20:08:58.052306 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:58.052772 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:58.052937 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:58.053689 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:58.065551 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.347s (SENT) I0507 20:08:58.065640 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:58.065920 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.442s (election complete) I0507 20:08:58.065999 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:58.274792 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:58.274933 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.244s (starting election) I0507 20:08:58.275028 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:58.275471 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:58.275650 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:58.276751 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:58.465373 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:58.465490 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.656s (starting election) I0507 20:08:58.465582 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:58.466069 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:58.466264 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:58.466594 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.888s (SENT) I0507 20:08:58.466683 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:58.466943 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.659s (election complete) I0507 20:08:58.467057 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:58.467259 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:58.509490 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:58.509634 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.331s (starting election) I0507 20:08:58.509804 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:58.510341 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:58.510545 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:58.511423 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:58.519994 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.244s (SENT) I0507 20:08:58.520045 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:58.520110 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:58.520164 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.903s (starting election) I0507 20:08:58.520274 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:58.520854 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:58.521056 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:58.521279 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.167s (election complete) I0507 20:08:58.521373 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:58.521986 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:58.606637 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.552s (SENT) I0507 20:08:58.606771 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:58.607143 1231 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.883s (election complete) I0507 20:08:58.607242 1231 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:58.688880 1231 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:58.688997 1231 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.155s (starting election) I0507 20:08:58.689108 1231 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:58.689596 1231 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:58.689792 1231 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:58.690564 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:58.842803 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.331s (SENT) I0507 20:08:58.842977 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:58.843365 1231 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.169s (election complete) I0507 20:08:58.843510 1231 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:58.845119 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.155s (SENT) I0507 20:08:58.845250 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:58.845255 1231 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:58.845342 1231 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.621s (starting election) I0507 20:08:58.845419 1231 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:58.845860 1231 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:58.845921 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.147s (election complete) I0507 20:08:58.846004 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:08:58.846046 1231 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:58.846922 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:58.994549 1231 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:58.994666 1231 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.091s (starting election) I0507 20:08:58.994771 1231 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:08:58.995262 1231 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:58.995455 1231 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:08:58.996170 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:59.013622 1231 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:59.013775 1231 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (starting election) I0507 20:08:59.013875 1231 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:59.014359 1231 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:59.014564 1231 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:59.015420 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:08:59.086443 1231 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:59.086544 1231 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.663s (starting election) I0507 20:08:59.086629 1231 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:08:59.086994 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.091s (SENT) I0507 20:08:59.087074 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:59.087134 1231 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:08:59.087343 1231 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:59.087499 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.379s (election complete) I0507 20:08:59.087560 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:59.088238 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:59.123471 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.656s (SENT) I0507 20:08:59.123585 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:08:59.123874 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.988s (election complete) I0507 20:08:59.123957 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:08:59.425295 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.903s (SENT) I0507 20:08:59.425453 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:59.425912 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.535s (election complete) I0507 20:08:59.426019 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:59.468974 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.621s (SENT) I0507 20:08:59.469101 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:59.469467 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.779s (election complete) I0507 20:08:59.469544 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:59.659469 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:08:56.660468 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:08:56.660738 (+ 270us) negotiation.cc:98] Waiting for socket to connect 0507 20:08:56.660753 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:08:56.660845 (+ 92us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:08:59.659173 (+2998328us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":78} W0507 20:08:59.659741 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:08:59.730455 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:08:59.730594 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.238s (starting election) I0507 20:08:59.730706 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:08:59.731158 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:08:59.731339 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:08:59.731578 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.716s (SENT) I0507 20:08:59.731665 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:59.731833 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.453s (election complete) I0507 20:08:59.731900 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:59.732620 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:08:59.751385 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.663s (SENT) I0507 20:08:59.751528 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:59.751891 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.675s (election complete) I0507 20:08:59.751984 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:08:59.969651 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.238s (SENT) I0507 20:08:59.969782 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:08:59.970134 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.358s (election complete) I0507 20:08:59.970237 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:00.112494 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:00.112676 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.983s (starting election) I0507 20:09:00.112812 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:00.113443 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:00.113693 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:00.115002 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:00.328820 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:00.328936 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.250s (starting election) I0507 20:09:00.329042 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:00.329515 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:00.329697 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:00.330461 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:00.427711 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:00.427896 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.217s (starting election) I0507 20:09:00.428035 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:00.428745 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:00.429008 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:00.429836 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:00.579316 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:00.579479 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.297s (starting election) I0507 20:09:00.579604 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:00.580006 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:00.580190 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:00.580359 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.250s (SENT) I0507 20:09:00.580437 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:00.580668 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.194s (election complete) I0507 20:09:00.580776 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:00.581148 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:00.646250 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:00.646399 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.266s (starting election) I0507 20:09:00.646502 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:00.647025 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:00.647235 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:00.647290 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.217s (SENT) I0507 20:09:00.647471 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:00.647838 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.556s (election complete) I0507 20:09:00.647960 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:00.648438 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:00.776293 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:00.776433 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.442s (starting election) I0507 20:09:00.776612 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:00.777189 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:00.777406 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:00.779731 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:00.878182 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.297s (SENT) I0507 20:09:00.878370 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:00.878866 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.460s (election complete) I0507 20:09:00.879040 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:00.915361 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.266s (SENT) I0507 20:09:00.915545 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:00.916134 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.965s (election complete) I0507 20:09:00.916275 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:01.096556 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:01.096717 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.129s (starting election) I0507 20:09:01.096859 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:01.097347 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:01.097565 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:01.097972 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.983s (SENT) I0507 20:09:01.098067 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:01.098412 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.922s (election complete) I0507 20:09:01.098520 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:01.098778 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:01.219931 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.442s (SENT) I0507 20:09:01.220067 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:01.220564 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.873s (election complete) I0507 20:09:01.220679 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:01.227864 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.129s (SENT) I0507 20:09:01.228036 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:01.228451 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.344s (election complete) I0507 20:09:01.228572 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:01.573484 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:01.573643 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.877s (starting election) I0507 20:09:01.573781 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:01.574483 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:01.574744 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:01.575565 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:01.882189 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:01.882375 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.733s (starting election) I0507 20:09:01.882504 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:01.883204 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:01.883452 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:01.884308 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:02.094384 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:02.094568 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.728s (starting election) I0507 20:09:02.094707 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:02.095490 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:02.095774 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:02.096999 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:02.451808 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:02.452049 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.209s (starting election) I0507 20:09:02.452188 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:02.452862 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:02.453146 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:02.453122 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.877s (SENT) I0507 20:09:02.453336 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:02.453745 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.480s (election complete) I0507 20:09:02.453836 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:02.454212 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:02.615820 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:02.615942 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.485s (starting election) I0507 20:09:02.616080 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:02.616562 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:02.616747 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:02.617166 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.733s (SENT) I0507 20:09:02.617318 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:02.617575 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.679s (election complete) I0507 20:09:02.617686 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:02.617700 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:02.663046 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.209s (SENT) I0507 20:09:02.663225 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:02.663712 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.491s (election complete) I0507 20:09:02.663875 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:02.823664 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:02.823822 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.508s (starting election) I0507 20:09:02.823927 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:02.824420 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:02.824609 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:02.824765 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.728s (SENT) I0507 20:09:02.824894 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:02.825327 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.123s (election complete) I0507 20:09:02.825466 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:02.825731 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:02.949733 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:02.949883 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.594s (starting election) I0507 20:09:02.950089 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:02.950531 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:02.950737 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:02.951751 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:03.102872 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.485s (SENT) I0507 20:09:03.103086 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:03.103579 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.241s (election complete) I0507 20:09:03.103710 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:03.156239 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:03.156402 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.717s (starting election) I0507 20:09:03.156517 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:03.157011 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:03.157196 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:03.158140 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:03.231640 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:00.230845 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:00.231087 (+ 242us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:00.231099 (+ 12us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:00.231255 (+ 156us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:03.231311 (+3000056us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":87} W0507 20:09:03.231943 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:09:03.335062 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.508s (SENT) I0507 20:09:03.335204 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:03.335644 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.595s (election complete) I0507 20:09:03.335757 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:03.345269 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:03.345383 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.702s (starting election) I0507 20:09:03.345484 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:03.345952 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:03.346127 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:03.347137 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:03.545465 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.594s (SENT) I0507 20:09:03.545620 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:03.545964 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.172s (election complete) I0507 20:09:03.546901 1226 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:03.719110 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:03.719319 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.315s (starting election) I0507 20:09:03.719427 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:03.719961 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:03.720221 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:03.721223 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:03.873908 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:03.874078 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.076s (starting election) I0507 20:09:03.874203 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:03.874683 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:03.874946 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:03.875244 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.717s (SENT) I0507 20:09:03.875325 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:03.875535 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.728s (election complete) I0507 20:09:03.875602 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:03.875807 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:03.952242 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.076s (SENT) I0507 20:09:03.952374 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:03.952806 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (election complete) I0507 20:09:03.952973 1226 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:04.036550 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.315s (SENT) I0507 20:09:04.036553 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:04.036711 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:04.036779 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.897s (starting election) I0507 20:09:04.036939 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:04.037745 1239 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.243s (election complete) I0507 20:09:04.037776 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:04.037888 1239 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:04.038152 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:04.039290 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:04.048851 1226 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:04.048993 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.253s (starting election) I0507 20:09:04.049118 1226 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:04.049343 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.702s (SENT) I0507 20:09:04.049499 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:04.049774 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:04.049826 1239 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.289s (election complete) I0507 20:09:04.049924 1239 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:04.050060 1226 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:04.050899 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:04.274729 1226 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:04.274936 1226 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.651s (starting election) I0507 20:09:04.275066 1226 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:04.275696 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:04.275913 1226 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:04.276952 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:04.281846 1226 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:04.281971 1226 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.857s (starting election) I0507 20:09:04.282099 1226 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:04.282719 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:04.282985 1226 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:04.283939 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:04.303364 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.253s (SENT) I0507 20:09:04.303481 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:04.303917 1226 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.900s (election complete) I0507 20:09:04.304082 1226 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:04.927001 1240 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:04.927096 1240 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.110s (starting election) I0507 20:09:04.927173 1240 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:04.927577 1240 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:04.927760 1240 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:04.927858 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.651s (SENT) I0507 20:09:04.927980 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:04.928207 1240 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.660s (election complete) I0507 20:09:04.928259 1240 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:04.928772 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:04.937125 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.897s (SENT) I0507 20:09:04.937239 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:04.937566 1240 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.985s (election complete) I0507 20:09:04.937656 1240 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:05.038980 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.110s (SENT) I0507 20:09:05.039089 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:05.039461 1240 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.371s (election complete) I0507 20:09:05.039575 1240 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:05.141114 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.857s (SENT) I0507 20:09:05.141248 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:05.141680 1240 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.277s (election complete) I0507 20:09:05.141785 1240 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:05.204490 1240 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:05.204593 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.225s (starting election) I0507 20:09:05.204674 1240 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:05.205149 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:05.205315 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:05.206251 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:05.411761 1240 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:05.411897 1240 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.454s (starting election) I0507 20:09:05.411978 1240 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:05.412400 1240 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:05.412559 1240 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:05.413647 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:05.419513 1240 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:05.419682 1240 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.246s (starting election) I0507 20:09:05.419787 1240 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:05.420445 1240 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:05.420697 1240 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:05.421842 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:05.430797 1240 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:05.430979 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.583s (starting election) I0507 20:09:05.431076 1240 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:05.431573 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.225s (SENT) I0507 20:09:05.431661 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:05.431702 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:05.431913 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:05.432060 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.713s (election complete) I0507 20:09:05.432190 1240 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:05.432915 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:05.666709 1240 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:05.666846 1240 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.439s (starting election) I0507 20:09:05.666993 1240 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:05.667438 1240 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:05.667621 1240 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:05.667835 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.246s (SENT) I0507 20:09:05.667901 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:05.668045 1240 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.918s (election complete) I0507 20:09:05.668089 1240 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:05.668731 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:05.866897 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.454s (SENT) I0507 20:09:05.867043 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:05.867298 1240 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:05.867403 1240 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.620s (starting election) I0507 20:09:05.867502 1240 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:05.867961 1240 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:05.868007 1242 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.824s (election complete) I0507 20:09:05.868077 1242 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:05.868134 1240 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:05.869045 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:06.017035 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.583s (SENT) I0507 20:09:06.017134 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:06.017482 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.123s (election complete) I0507 20:09:06.017594 1240 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:06.107513 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.439s (SENT) I0507 20:09:06.107617 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:06.107961 1240 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.171s (election complete) I0507 20:09:06.108072 1240 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:06.141263 1240 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:06.141371 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.391s (starting election) I0507 20:09:06.141453 1240 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:06.141865 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:06.141995 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:06.142971 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:06.279911 1240 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:06.280068 1240 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.966s (starting election) I0507 20:09:06.280195 1240 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:06.280915 1240 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:06.281126 1240 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:06.282239 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:06.489545 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.620s (SENT) I0507 20:09:06.489665 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:06.489979 1240 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.673s (election complete) I0507 20:09:06.490057 1240 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:06.533502 1240 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:06.533625 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.719s (starting election) I0507 20:09:06.533710 1240 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:06.534153 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:06.534235 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.391s (SENT) I0507 20:09:06.534312 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:06.534317 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:06.534554 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.189s (election complete) I0507 20:09:06.534611 1240 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:06.535161 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:06.724341 1240 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:06.724478 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.454s (starting election) I0507 20:09:06.724578 1240 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:06.725191 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:06.725412 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:06.726358 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:06.807684 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:03.809276 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:03.809643 (+ 367us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:03.809662 (+ 19us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:03.809790 (+ 128us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:06.807312 (+2997522us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":111} W0507 20:09:06.808053 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:07.163729 1240 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:07.163872 1240 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.802s (starting election) I0507 20:09:07.163960 1240 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:07.164448 1240 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:07.164629 1240 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:07.165592 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:07.179651 1240 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:07.179780 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.170s (starting election) I0507 20:09:07.179867 1240 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:07.180176 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.454s (SENT) I0507 20:09:07.180243 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:07.180379 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:07.180554 1240 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:07.180661 1240 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.870s (election complete) I0507 20:09:07.180703 1240 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:07.181520 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:07.246512 1244 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:07.246644 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.212s (starting election) I0507 20:09:07.246739 1244 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:07.247298 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:07.247486 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:07.248280 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.966s (SENT) I0507 20:09:07.248378 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:09:07.248531 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:07.248611 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.353s (election complete) I0507 20:09:07.248728 1244 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:07.254950 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.719s (SENT) I0507 20:09:07.255041 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:07.255309 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.084s (election complete) I0507 20:09:07.255378 1244 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:07.340147 1244 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:07.340281 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.171s (starting election) I0507 20:09:07.340361 1244 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:07.340850 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:07.341069 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:07.342093 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:07.351904 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.170s (SENT) I0507 20:09:07.352012 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:07.352354 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.669s (election complete) I0507 20:09:07.352440 1244 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:07.460265 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.212s (SENT) I0507 20:09:07.460381 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:07.460707 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.669s (election complete) I0507 20:09:07.460785 1244 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:07.512758 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.171s (SENT) I0507 20:09:07.512873 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:07.513206 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.256s (election complete) I0507 20:09:07.513278 1244 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:07.769672 1244 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:07.769836 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.260s (starting election) I0507 20:09:07.769951 1244 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:07.770681 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:07.770988 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:07.771978 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:07.966630 1244 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:07.966799 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.354s (starting election) I0507 20:09:07.966953 1244 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:07.967767 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:07.967874 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.802s (SENT) I0507 20:09:07.967972 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:07.968034 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:07.968209 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (election complete) I0507 20:09:07.968287 1244 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:07.969359 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:08.031087 1246 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:08.031216 1246 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.768s (starting election) I0507 20:09:08.031301 1246 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:08.031690 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.260s (SENT) I0507 20:09:08.031757 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.031780 1246 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:08.031950 1246 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:08.032032 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.265s (election complete) I0507 20:09:08.032146 1244 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:08.033231 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:08.131103 1244 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:08.131233 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.207s (starting election) I0507 20:09:08.131322 1244 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:08.131808 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:08.132038 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:08.133138 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:08.290621 1244 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:08.290747 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.588s (starting election) I0507 20:09:08.290830 1244 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:08.291362 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:08.291544 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:08.292469 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:08.298267 1244 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:08.298379 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.336s (starting election) I0507 20:09:08.298462 1244 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:08.298946 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:08.299113 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:08.299948 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:08.322791 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.354s (SENT) I0507 20:09:08.322968 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.323293 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (election complete) I0507 20:09:08.323367 1244 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:08.339012 1244 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:08.339133 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.389s (starting election) I0507 20:09:08.339220 1244 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } W0507 20:09:08.339452 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.207s (SENT) I0507 20:09:08.339512 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.339668 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:08.339783 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:08.339764 1246 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.919s (election complete) I0507 20:09:08.339856 1246 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:08.340559 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:08.635046 1246 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:08.635197 1246 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.349s (starting election) I0507 20:09:08.635303 1246 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:08.635957 1246 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:08.635982 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.336s (SENT) I0507 20:09:08.636034 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.636173 1246 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:08.636298 1246 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.785s (election complete) I0507 20:09:08.636338 1246 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:08.637169 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:08.644775 1244 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:08.644878 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.080s (starting election) I0507 20:09:08.644955 1244 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:08.645397 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:08.645568 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:08.646512 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:08.725580 1244 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:08.725710 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.610s (starting election) I0507 20:09:08.725796 1244 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:08.726287 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:08.726472 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:08.726595 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.080s (SENT) I0507 20:09:08.726699 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.727066 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.362s (election complete) I0507 20:09:08.727180 1244 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:08.727290 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:08.729806 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.389s (SENT) I0507 20:09:08.729894 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.730134 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.270s (election complete) I0507 20:09:08.730183 1244 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:08.801180 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.768s (SENT) I0507 20:09:08.801303 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.801676 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.649s (election complete) I0507 20:09:08.801784 1244 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:08.881827 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.588s (SENT) I0507 20:09:08.881929 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.882318 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.841s (election complete) I0507 20:09:08.882433 1244 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:08.985658 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.349s (SENT) I0507 20:09:08.985759 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:08.986050 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.563s (election complete) I0507 20:09:08.986148 1244 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:09.000702 1244 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:09.000831 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.270s (starting election) I0507 20:09:09.000916 1244 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:09.001394 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:09.001639 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:09.002425 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:09.271319 1244 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:09.271464 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.512s (starting election) I0507 20:09:09.271549 1244 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:09.272078 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:09.272259 1244 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:09.272542 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.270s (SENT) I0507 20:09:09.272626 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:09.273001 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.557s (election complete) I0507 20:09:09.273084 1244 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:09.273242 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:09.338382 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.610s (SENT) I0507 20:09:09.338495 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:09.338857 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.880s (election complete) I0507 20:09:09.338973 1244 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:09.550557 1244 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:09.550683 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.754s (starting election) I0507 20:09:09.550774 1244 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:09.551264 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:09.551442 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:09.552399 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:09.786242 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.512s (SENT) I0507 20:09:09.786361 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:09.786737 1244 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.887s (election complete) I0507 20:09:09.786842 1244 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:10.220027 1244 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:10.220158 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.084s (starting election) I0507 20:09:10.220240 1244 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:10.220742 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:10.221014 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:10.222146 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:10.304862 1244 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:10.304993 1244 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.671s (starting election) W0507 20:09:10.305270 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.084s (SENT) I0507 20:09:10.305096 1244 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:10.305356 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:10.305660 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:10.305907 1244 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:10.306020 1244 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:10.306037 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.323s (election complete) I0507 20:09:10.306057 1244 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.451s (starting election) I0507 20:09:10.306159 1244 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:10.306622 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:10.306766 1244 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:10.306967 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:10.307090 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:10.307109 1249 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:10.307242 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.754s (SENT) I0507 20:09:10.307294 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:10.307462 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.451s (election complete) I0507 20:09:10.307512 1249 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:10.395630 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:07.397515 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:07.397835 (+ 320us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:07.397853 (+ 18us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:07.397983 (+ 130us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:10.395260 (+2997277us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":103} W0507 20:09:10.395959 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:10.631413 1249 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:10.631562 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.614s (starting election) I0507 20:09:10.631654 1249 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:10.632244 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:10.632477 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:10.633568 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:10.674271 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:10.674427 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.517s (starting election) I0507 20:09:10.674535 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:10.675220 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:10.675451 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:10.676789 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:10.758364 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.451s (SENT) I0507 20:09:10.758500 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:10.758903 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.650s (election complete) I0507 20:09:10.759016 1249 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:10.979429 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.671s (SENT) I0507 20:09:10.979542 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:10.979848 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.239s (election complete) I0507 20:09:10.979928 1249 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:11.192001 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:11.192119 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.571s (starting election) I0507 20:09:11.192194 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:11.192628 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:11.192795 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:11.193784 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:11.194231 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.517s (SENT) I0507 20:09:11.194278 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:11.194490 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.631s (election complete) I0507 20:09:11.194540 1249 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:11.219617 1249 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:11.219722 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.936s (starting election) I0507 20:09:11.219797 1249 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:11.220212 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:11.220377 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:11.221532 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:11.247123 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.614s (SENT) I0507 20:09:11.247239 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:11.247555 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.491s (election complete) I0507 20:09:11.247642 1249 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:11.409942 1249 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:11.410063 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.704s (starting election) I0507 20:09:11.410153 1249 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:11.410717 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:11.410953 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:11.411875 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:11.739332 1249 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:11.739439 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.305s (starting election) I0507 20:09:11.739512 1249 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:11.740012 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:11.740164 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:11.741080 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:11.764303 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.571s (SENT) I0507 20:09:11.764420 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:11.764827 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.753s (election complete) I0507 20:09:11.764940 1249 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:12.045547 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.305s (SENT) I0507 20:09:12.045658 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:12.045851 1249 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:12.045979 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.299s (starting election) I0507 20:09:12.046044 1249 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:12.046438 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:12.046455 1253 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.368s (election complete) I0507 20:09:12.046530 1253 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:12.046574 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:12.047415 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:12.115489 1249 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:12.115694 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.901s (starting election) I0507 20:09:12.115785 1249 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:12.116377 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:12.116602 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:12.116629 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.704s (SENT) I0507 20:09:12.116690 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:09:12.117457 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:12.117483 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.787s (election complete) I0507 20:09:12.117518 1249 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:12.157761 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.936s (SENT) I0507 20:09:12.157848 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:12.158182 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.278s (election complete) I0507 20:09:12.158300 1249 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:12.346285 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.299s (SENT) I0507 20:09:12.346407 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:12.346828 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.509s (election complete) I0507 20:09:12.346961 1249 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:12.519327 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:12.519459 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.526s (starting election) I0507 20:09:12.519567 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:12.520182 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:12.520359 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:12.521200 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:12.857235 1249 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:12.857354 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.565s (starting election) I0507 20:09:12.857481 1249 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:12.858258 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:12.858439 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:12.859259 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:12.905714 1249 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:12.905844 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.855s (starting election) I0507 20:09:12.905939 1249 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:12.906519 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:12.906726 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:12.907779 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:13.019764 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.901s (SENT) I0507 20:09:13.019877 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:13.020229 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.968s (election complete) I0507 20:09:13.020359 1249 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:13.045958 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:13.046087 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.791s (starting election) I0507 20:09:13.046169 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:13.046720 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:13.046970 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:13.047207 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.526s (SENT) I0507 20:09:13.047288 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:13.047569 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.661s (election complete) I0507 20:09:13.047617 1249 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:13.048035 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:13.423655 1249 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:13.423765 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.896s (starting election) I0507 20:09:13.423867 1249 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:13.424340 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:13.424501 1249 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:13.424775 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.565s (SENT) I0507 20:09:13.424834 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:13.425034 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.814s (election complete) I0507 20:09:13.425091 1249 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:13.425412 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:13.709300 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:13.709472 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.114s (starting election) I0507 20:09:13.709589 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:13.710135 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:13.710340 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:13.711433 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:13.763011 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.855s (SENT) I0507 20:09:13.763244 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:13.763692 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.474s (election complete) I0507 20:09:13.763787 1249 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:13.824738 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:13.824792 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.114s (SENT) I0507 20:09:13.824859 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.442s (starting election) I0507 20:09:13.824924 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:13.824986 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:13.825515 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:13.825520 1254 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.096s (election complete) I0507 20:09:13.825593 1254 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:13.825750 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:13.826588 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:13.839901 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.791s (SENT) I0507 20:09:13.840054 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:13.840450 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.114s (election complete) I0507 20:09:13.840584 1249 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:13.955919 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:13.956104 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.639s (starting election) I0507 20:09:13.956251 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:13.956748 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:13.956945 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:13.957974 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:13.995546 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:10.995203 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:10.995508 (+ 305us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:10.995521 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:10.995626 (+ 105us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:13.995218 (+2999592us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":67} W0507 20:09:13.995826 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:14.239112 1249 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:14.239234 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.327s (starting election) I0507 20:09:14.239327 1249 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:14.239651 1254 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:14.239761 1254 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.392s (starting election) I0507 20:09:14.239858 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:14.239848 1254 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:14.240236 1254 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:14.240384 1254 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:14.240528 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:14.240813 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:14.240952 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:14.268579 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.442s (SENT) I0507 20:09:14.268714 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:14.269071 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.791s (election complete) I0507 20:09:14.269174 1249 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:14.322536 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.896s (SENT) I0507 20:09:14.322657 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:14.323015 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.658s (election complete) I0507 20:09:14.323138 1249 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:14.567183 1249 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:14.567322 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.179s (starting election) I0507 20:09:14.567426 1249 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:14.567894 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:14.568064 1249 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:14.568244 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.327s (SENT) I0507 20:09:14.568331 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:14.568579 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.310s (election complete) I0507 20:09:14.568645 1249 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:14.569120 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:14.597945 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.639s (SENT) I0507 20:09:14.598088 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:14.598466 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.418s (election complete) I0507 20:09:14.598552 1249 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:14.633812 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.392s (SENT) I0507 20:09:14.633932 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:14.634291 1249 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.980s (election complete) I0507 20:09:14.634380 1249 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:14.748505 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.179s (SENT) I0507 20:09:14.748656 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:14.749135 1249 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.932s (election complete) I0507 20:09:14.749272 1249 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:15.017019 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:15.017138 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.303s (starting election) I0507 20:09:15.017251 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:15.017727 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:15.017935 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:15.018973 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:15.321595 1249 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:15.321637 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.303s (SENT) I0507 20:09:15.321777 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.336s (starting election) I0507 20:09:15.321830 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:15.321892 1249 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:15.322487 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:15.322681 1249 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:15.322827 1249 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.956s (election complete) I0507 20:09:15.322937 1249 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:15.323565 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:15.615186 1257 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:15.615298 1257 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.531s (starting election) I0507 20:09:15.615418 1257 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:15.615826 1257 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:15.615978 1257 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:15.616658 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:15.659906 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.336s (SENT) I0507 20:09:15.660049 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:15.660372 1257 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.643s (election complete) I0507 20:09:15.660492 1257 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:15.682478 1257 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:15.682629 1257 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.913s (starting election) I0507 20:09:15.682741 1257 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:15.683248 1257 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:15.683441 1257 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:15.684574 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:16.147588 1257 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:16.147737 1257 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.631s (starting election) I0507 20:09:16.147900 1257 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:16.148356 1257 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:16.148547 1257 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:16.148903 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.531s (SENT) I0507 20:09:16.149006 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:16.149230 1257 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.189s (election complete) I0507 20:09:16.149296 1257 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:16.149659 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:16.304283 1257 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:16.304405 1257 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.594s (starting election) I0507 20:09:16.304487 1257 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:16.304934 1257 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:16.305107 1257 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:16.306012 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:16.339423 1257 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:16.339562 1257 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.403s (starting election) I0507 20:09:16.339646 1257 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:16.340142 1257 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:16.340306 1257 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:16.341035 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:16.596925 1257 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:16.597081 1257 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.279s (starting election) I0507 20:09:16.597165 1257 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:16.597473 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.913s (SENT) I0507 20:09:16.597543 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:16.597599 1257 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:16.597784 1257 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:16.597946 1257 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.922s (election complete) I0507 20:09:16.597980 1257 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:16.598573 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:16.743858 1258 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:16.743978 1258 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.511s (starting election) W0507 20:09:16.744149 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.403s (SENT) I0507 20:09:16.744068 1258 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:16.744230 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:16.744504 1258 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:16.744516 1257 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.282s (election complete) I0507 20:09:16.744571 1257 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:16.744668 1258 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:16.745602 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:16.781036 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.631s (SENT) I0507 20:09:16.781131 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:16.781395 1258 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.675s (election complete) I0507 20:09:16.781463 1258 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:16.878140 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.279s (SENT) I0507 20:09:16.878258 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:16.878638 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.373s (election complete) I0507 20:09:16.878741 1258 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:16.899674 1258 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:16.899791 1258 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.947s (starting election) I0507 20:09:16.899869 1258 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:16.900319 1258 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:16.900475 1258 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:16.901239 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.594s (SENT) I0507 20:09:16.901337 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:16.901533 1258 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.784s (election complete) W0507 20:09:16.901536 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:16.901583 1258 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:17.252476 1258 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:17.252598 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.427s (starting election) I0507 20:09:17.252691 1258 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:17.253155 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:17.253326 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:17.254212 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:17.256690 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.511s (SENT) I0507 20:09:17.256772 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:17.257047 1258 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.935s (election complete) I0507 20:09:17.257114 1258 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:17.603622 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:14.602321 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:14.602549 (+ 228us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:14.602564 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:14.602659 (+ 95us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:17.603233 (+3000574us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":75} W0507 20:09:17.603935 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:17.680636 1258 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:17.680768 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.795s (starting election) I0507 20:09:17.680863 1258 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:17.681377 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:17.681432 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.427s (SENT) I0507 20:09:17.681509 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:17.681566 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:17.681773 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.593s (election complete) I0507 20:09:17.681825 1258 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:17.682373 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:17.686507 1258 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:17.686594 1258 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.404s (starting election) I0507 20:09:17.686673 1258 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:17.687141 1258 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:17.687363 1258 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:17.688184 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:17.849158 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.947s (SENT) I0507 20:09:17.849259 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:17.849612 1258 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.287s (election complete) I0507 20:09:17.849728 1258 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:18.091630 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.404s (SENT) I0507 20:09:18.091720 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:18.092030 1258 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.711s (election complete) I0507 20:09:18.092129 1258 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:18.193146 1258 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:18.193281 1258 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.914s (starting election) I0507 20:09:18.193378 1258 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:18.193869 1258 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:18.194054 1258 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:18.195091 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:18.275713 1258 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:18.275830 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.833s (starting election) I0507 20:09:18.275912 1258 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:18.276361 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:18.276538 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:18.277400 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:18.478451 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.795s (SENT) I0507 20:09:18.478534 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:18.478816 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.940s (election complete) I0507 20:09:18.478948 1258 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:18.803828 1258 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:18.803972 1258 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.382s (starting election) I0507 20:09:18.804057 1258 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:18.804574 1258 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:18.804757 1258 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:18.805807 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:19.108422 1258 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:19.108551 1258 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.336s (starting election) I0507 20:09:19.108636 1258 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:19.109097 1258 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:19.109264 1258 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:19.109563 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.914s (SENT) I0507 20:09:19.109624 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:19.109789 1258 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.250s (election complete) I0507 20:09:19.109844 1258 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:19.110103 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:19.110528 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.833s (SENT) I0507 20:09:19.110575 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:19.110702 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.085s (election complete) I0507 20:09:19.110749 1258 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:19.186715 1258 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:19.186833 1258 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.719s (starting election) I0507 20:09:19.186961 1258 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:19.187378 1258 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:19.187532 1258 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:19.187754 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.382s (SENT) I0507 20:09:19.187826 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:19.187994 1258 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.984s (election complete) I0507 20:09:19.188030 1258 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:19.188362 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:19.196939 1258 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:19.197062 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.257s (starting election) I0507 20:09:19.197149 1258 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:19.197602 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:19.197780 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:19.198395 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:19.360816 1258 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:19.360950 1258 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.847s (starting election) I0507 20:09:19.361044 1258 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:19.361568 1258 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:19.361757 1258 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:19.362452 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:19.445767 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.336s (SENT) I0507 20:09:19.445878 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:19.446183 1258 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.380s (election complete) I0507 20:09:19.446281 1258 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:19.454810 1258 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:19.454931 1258 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.203s (starting election) I0507 20:09:19.455024 1258 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:19.455404 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.257s (SENT) I0507 20:09:19.455473 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:19.455510 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:19.455710 1258 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:19.455942 1263 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.698s (election complete) I0507 20:09:19.455998 1263 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:19.456553 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:19.659821 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.203s (SENT) I0507 20:09:19.659970 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:19.660449 1263 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.365s (election complete) I0507 20:09:19.660549 1263 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:19.826874 1263 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:19.827023 1263 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.845s (starting election) I0507 20:09:19.827122 1263 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:19.827692 1263 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:19.827936 1263 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:19.829017 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:19.908628 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.719s (SENT) I0507 20:09:19.908723 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:19.909023 1263 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.339s (election complete) I0507 20:09:19.909103 1263 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:20.025872 1263 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:20.026036 1263 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.298s (starting election) I0507 20:09:20.026137 1263 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:20.026577 1263 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:20.026742 1263 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:20.027840 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:20.210307 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.847s (SENT) I0507 20:09:20.210412 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:20.210718 1263 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.152s (election complete) I0507 20:09:20.210803 1263 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:20.249378 1263 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:20.249516 1263 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.589s (starting election) I0507 20:09:20.249624 1263 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:20.250077 1263 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:20.250232 1263 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:20.251379 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:20.317106 1263 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:20.317229 1263 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.476s (starting election) I0507 20:09:20.317332 1263 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:20.317811 1263 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:20.317992 1263 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:20.319207 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:20.325069 1263 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:20.325184 1263 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.507s (starting election) I0507 20:09:20.325273 1263 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:20.325650 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.298s (SENT) I0507 20:09:20.325721 1263 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:20.325743 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:20.325912 1263 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:20.326045 1263 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.678s (election complete) I0507 20:09:20.326133 1263 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:20.326917 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:20.364136 1264 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:20.364253 1264 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.381s (starting election) I0507 20:09:20.364344 1264 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:20.364815 1264 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:20.364967 1264 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:20.365787 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:20.674269 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.845s (SENT) I0507 20:09:20.674393 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:20.674793 1264 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.220s (election complete) I0507 20:09:20.674981 1264 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:20.746975 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.381s (SENT) I0507 20:09:20.747081 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:20.747509 1264 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.359s (election complete) I0507 20:09:20.747635 1264 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:20.794353 1264 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:20.794378 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.476s (SENT) I0507 20:09:20.794512 1264 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.799s (starting election) I0507 20:09:20.794554 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:20.794651 1264 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:20.795220 1264 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:20.795229 1263 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.415s (election complete) I0507 20:09:20.795321 1263 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:20.795473 1264 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:20.796263 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:20.834764 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.507s (SENT) I0507 20:09:20.834951 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:20.835355 1264 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.075s (election complete) I0507 20:09:20.835458 1264 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:20.839857 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.589s (SENT) I0507 20:09:20.839962 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:20.840368 1264 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.319s (election complete) I0507 20:09:20.840481 1264 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:20.911324 1264 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:20.911448 1264 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.893s (starting election) I0507 20:09:20.911547 1264 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:20.912058 1264 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:20.912262 1264 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:20.913218 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:21.107811 1264 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:21.107934 1264 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.773s (starting election) I0507 20:09:21.108047 1264 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:21.108532 1264 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:21.108739 1264 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:21.109760 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:21.160497 1264 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:21.160663 1264 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.360s (starting election) I0507 20:09:21.160768 1264 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:21.161360 1264 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:21.161568 1264 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:21.162591 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:21.215628 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:18.217464 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:18.217740 (+ 276us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:18.217758 (+ 18us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:18.217888 (+ 130us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:21.215334 (+2997446us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":88} W0507 20:09:21.215965 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:21.521487 1264 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:21.521667 1264 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.112s (starting election) I0507 20:09:21.521811 1264 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:21.522447 1264 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:21.522606 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.360s (SENT) I0507 20:09:21.522712 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:21.522773 1264 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:21.523015 1264 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.646s (election complete) I0507 20:09:21.523088 1264 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:21.523910 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:21.595538 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.799s (SENT) I0507 20:09:21.595670 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:21.596040 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.456s (election complete) I0507 20:09:21.596154 1266 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:21.637037 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.112s (SENT) I0507 20:09:21.637185 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:21.637554 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.204s (election complete) I0507 20:09:21.637665 1266 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:21.805955 1266 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:21.806085 1266 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.452s (starting election) I0507 20:09:21.806226 1266 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:21.806772 1266 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:21.806908 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.893s (SENT) I0507 20:09:21.807041 1266 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:21.807065 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:21.807484 1266 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.843s (election complete) I0507 20:09:21.807612 1266 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:21.807948 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:21.843518 1266 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:21.843670 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.161s (starting election) I0507 20:09:21.843828 1266 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:21.844317 1266 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:21.844506 1266 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:21.845356 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:21.881505 1266 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:21.881651 1266 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.566s (starting election) I0507 20:09:21.881773 1266 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:21.882354 1266 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:21.882586 1266 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:21.882961 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.773s (SENT) I0507 20:09:21.883035 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:21.883383 1266 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.125s (election complete) I0507 20:09:21.883539 1266 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:21.883587 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:22.005208 1266 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:22.005336 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.899s (starting election) I0507 20:09:22.005439 1266 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:22.005951 1266 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:22.006132 1266 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:22.006420 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.161s (SENT) I0507 20:09:22.006516 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:22.007035 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.206s (election complete) W0507 20:09:22.007086 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:22.007112 1266 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:22.009747 1266 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:22.009827 1266 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.512s (starting election) I0507 20:09:22.009932 1266 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:22.010370 1266 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:22.010550 1266 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:22.011220 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:22.214399 1266 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:22.214546 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.082s (starting election) I0507 20:09:22.214673 1266 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:22.215232 1266 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:22.215423 1266 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:22.216364 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:22.260017 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.452s (SENT) I0507 20:09:22.260166 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:22.260658 1266 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.373s (election complete) I0507 20:09:22.260773 1266 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:22.297353 1266 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:22.297508 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.154s (starting election) I0507 20:09:22.297616 1266 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:22.298135 1266 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:22.298357 1266 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:22.298686 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.082s (SENT) I0507 20:09:22.298787 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:22.299093 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.772s (election complete) I0507 20:09:22.299221 1266 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:22.299338 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:22.449398 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.566s (SENT) I0507 20:09:22.449568 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:22.449878 1266 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.897s (election complete) I0507 20:09:22.449993 1266 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:22.453161 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.154s (SENT) I0507 20:09:22.453258 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:22.453532 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.785s (election complete) I0507 20:09:22.453631 1266 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:22.524210 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.512s (SENT) I0507 20:09:22.524312 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:22.524682 1266 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.978s (election complete) I0507 20:09:22.524806 1266 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:22.634915 1266 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:22.635062 1266 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.961s (starting election) I0507 20:09:22.635174 1266 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:22.635730 1266 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:22.635967 1266 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:22.636857 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:22.905781 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.899s (SENT) I0507 20:09:22.905905 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:22.906335 1266 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.744s (election complete) I0507 20:09:22.906456 1266 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:23.503648 1268 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:23.503762 1268 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.629s (starting election) I0507 20:09:23.503867 1268 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:23.504379 1268 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:23.504607 1268 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:23.505537 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:23.597013 1268 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:23.597139 1268 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.144s (starting election) I0507 20:09:23.597250 1268 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:23.597774 1268 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:23.597949 1268 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:23.598083 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.961s (SENT) I0507 20:09:23.598165 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:23.598376 1268 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.085s (election complete) I0507 20:09:23.598438 1268 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:23.598803 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:23.651357 1268 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:23.651510 1268 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.146s (starting election) I0507 20:09:23.651639 1268 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:23.652254 1268 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:23.652487 1268 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:23.653282 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:23.684828 1268 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:23.684965 1268 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.458s (starting election) I0507 20:09:23.685081 1268 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:23.685652 1268 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:23.685849 1268 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:23.686707 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:23.742998 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.144s (SENT) I0507 20:09:23.743124 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:23.743540 1268 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.622s (election complete) I0507 20:09:23.743665 1268 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:23.798866 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.146s (SENT) I0507 20:09:23.798866 1268 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:23.798988 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:23.799003 1268 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.893s (starting election) I0507 20:09:23.799139 1268 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:23.799662 1268 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:23.799839 1268 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:23.799998 1268 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.904s (election complete) I0507 20:09:23.800078 1268 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:23.800863 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:24.133718 1270 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:24.133862 1270 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.925s (starting election) I0507 20:09:24.133963 1270 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:24.134409 1270 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:24.134639 1270 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:24.134927 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.629s (SENT) I0507 20:09:24.135094 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:24.135447 1270 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.821s (election complete) I0507 20:09:24.135565 1270 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:24.135565 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:24.144592 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.458s (SENT) I0507 20:09:24.144750 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:24.145176 1270 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.599s (election complete) I0507 20:09:24.145320 1270 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:24.695041 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.893s (SENT) I0507 20:09:24.695138 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:24.695694 1271 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.919s (election complete) I0507 20:09:24.695771 1271 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:24.745008 1271 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:24.745132 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.282s (starting election) I0507 20:09:24.745244 1271 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:24.745719 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:24.745920 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:24.746930 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:24.843598 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:21.842627 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:21.842971 (+ 344us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:21.842987 (+ 16us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:21.843109 (+ 122us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:24.843268 (+3000159us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":91} W0507 20:09:24.843870 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:24.957453 1271 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:24.957576 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.438s (starting election) I0507 20:09:24.957666 1271 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:24.958155 1271 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:24.958358 1271 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:24.959518 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:25.028339 1271 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:25.028475 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.445s (starting election) I0507 20:09:25.028568 1271 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:25.029021 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:25.029187 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:25.029472 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.282s (SENT) I0507 20:09:25.029629 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:09:25.029877 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:25.029875 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.216s (election complete) I0507 20:09:25.030004 1271 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:25.062634 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.925s (SENT) I0507 20:09:25.062727 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:25.063045 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.851s (election complete) I0507 20:09:25.063130 1271 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:25.247306 1271 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:25.247467 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.273s (starting election) I0507 20:09:25.247604 1271 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:25.248236 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:25.248523 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:25.249492 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:25.396798 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.438s (SENT) I0507 20:09:25.396919 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:25.397294 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.137s (election complete) I0507 20:09:25.397419 1271 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:25.475170 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.445s (SENT) I0507 20:09:25.475265 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:25.475558 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.353s (election complete) I0507 20:09:25.475646 1271 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:25.522289 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.273s (SENT) I0507 20:09:25.522416 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:25.522792 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.942s (election complete) I0507 20:09:25.522913 1271 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:25.535158 1271 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:25.535290 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.521s (starting election) I0507 20:09:25.535413 1271 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:25.535851 1271 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:25.536033 1271 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:25.536993 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:25.615916 1271 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:25.616065 1271 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.896s (starting election) I0507 20:09:25.616163 1271 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:25.616709 1271 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:25.616909 1271 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:25.617815 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:26.057067 1271 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:26.057235 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.752s (starting election) I0507 20:09:26.057345 1271 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:26.057847 1271 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:26.058049 1271 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:26.058452 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.521s (SENT) I0507 20:09:26.058573 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:26.058811 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.706s (election complete) W0507 20:09:26.058818 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:26.058923 1271 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:26.465911 1271 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:26.466035 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.079s (starting election) I0507 20:09:26.466133 1271 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:26.466603 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:26.466786 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:26.467628 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:26.513155 1271 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:26.513272 1271 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.360s (starting election) I0507 20:09:26.513377 1271 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:26.513795 1271 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:26.513985 1271 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:26.514648 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.896s (SENT) I0507 20:09:26.514777 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:09:26.514900 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:26.515015 1271 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.523s (election complete) I0507 20:09:26.515086 1271 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:26.545850 1271 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:26.545970 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.637s (starting election) I0507 20:09:26.546072 1271 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:26.546571 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:26.546756 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:26.546897 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.079s (SENT) I0507 20:09:26.547004 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:26.547247 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.823s (election complete) I0507 20:09:26.547380 1271 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:26.547708 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:26.765523 1271 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:26.765679 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.720s (starting election) I0507 20:09:26.765790 1271 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:26.766449 1271 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:26.766717 1271 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:26.767626 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:26.811341 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.752s (SENT) I0507 20:09:26.811444 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:26.811820 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.903s (election complete) I0507 20:09:26.811969 1271 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:26.874387 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.360s (SENT) I0507 20:09:26.874514 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:26.874928 1271 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.433s (election complete) I0507 20:09:26.875038 1271 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:27.186251 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.637s (SENT) I0507 20:09:27.186363 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:27.186692 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.272s (election complete) I0507 20:09:27.186791 1271 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:27.309346 1271 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:27.309469 1271 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.635s (starting election) I0507 20:09:27.309550 1271 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:27.310019 1271 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:27.310191 1271 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:27.311262 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:27.460064 1271 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:27.460161 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.605s (starting election) I0507 20:09:27.460261 1271 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:27.460675 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:27.460836 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:27.461741 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:27.488338 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.720s (SENT) I0507 20:09:27.488428 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:27.488747 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.590s (election complete) I0507 20:09:27.488855 1271 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:27.945632 1271 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:27.945775 1271 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.847s (starting election) I0507 20:09:27.945863 1271 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:27.946363 1271 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:27.946554 1271 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:27.946823 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.635s (SENT) I0507 20:09:27.946925 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:27.947165 1271 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.471s (election complete) I0507 20:09:27.947239 1271 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:27.947513 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:28.066440 1271 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:28.066610 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.872s (starting election) I0507 20:09:28.066710 1271 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:28.067155 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.605s (SENT) I0507 20:09:28.067239 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:28.067258 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:28.067430 1271 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:28.067561 1271 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.320s (election complete) I0507 20:09:28.067618 1271 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:28.068307 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:28.079454 1276 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:28.079583 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.167s (starting election) I0507 20:09:28.079692 1276 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:28.080261 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:28.080462 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:28.081285 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:28.247887 1276 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:28.248010 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.167s (SENT) I0507 20:09:28.248026 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.391s (starting election) I0507 20:09:28.248098 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:28.248194 1276 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:28.248735 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:28.248762 1271 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.472s (election complete) I0507 20:09:28.248828 1271 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:28.248921 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:28.249682 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:28.388198 1276 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:28.388324 1276 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.602s (starting election) I0507 20:09:28.388417 1276 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:28.388931 1276 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:28.389117 1276 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:28.389992 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:28.418656 1276 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:28.418787 1276 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.526s (starting election) I0507 20:09:28.418867 1276 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:28.419330 1276 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:28.419492 1276 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:28.420312 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:28.475575 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:25.475553 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:25.475883 (+ 330us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:25.475900 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:25.476026 (+ 126us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:28.475269 (+2999243us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":124} W0507 20:09:28.475891 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:09:28.640786 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.391s (SENT) I0507 20:09:28.640892 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:28.641181 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.130s (election complete) I0507 20:09:28.641260 1276 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:28.771841 1276 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:28.771960 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.844s (starting election) I0507 20:09:28.772054 1276 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:28.772480 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:28.772634 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:28.773375 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:28.794385 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.847s (SENT) I0507 20:09:28.794487 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:28.794801 1276 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.837s (election complete) I0507 20:09:28.794912 1276 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:28.941326 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.872s (SENT) I0507 20:09:28.941421 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:28.941730 1276 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.941s (election complete) I0507 20:09:28.941833 1276 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:28.947042 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.526s (SENT) I0507 20:09:28.947124 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:28.947383 1276 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.373s (election complete) I0507 20:09:28.947468 1276 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:28.992671 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.602s (SENT) I0507 20:09:28.992784 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:28.993104 1276 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.944s (election complete) I0507 20:09:28.993207 1276 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:29.320866 1276 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:29.320977 1276 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.783s (starting election) I0507 20:09:29.321058 1276 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:29.321461 1276 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:29.321594 1276 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:29.322547 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:29.616987 1276 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:29.617100 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.320s (starting election) I0507 20:09:29.617192 1276 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:29.617597 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:29.617750 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:29.617791 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.844s (SENT) I0507 20:09:29.617871 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:29.618052 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.894s (election complete) I0507 20:09:29.618121 1276 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:29.618712 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:29.938210 1276 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:29.938325 1276 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.404s (starting election) I0507 20:09:29.938403 1276 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:29.938800 1276 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:29.938935 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.320s (SENT) I0507 20:09:29.938980 1276 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:29.939046 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:29.939265 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.388s (election complete) I0507 20:09:29.939306 1276 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:29.939755 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:30.104704 1276 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:30.104820 1276 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.624s (starting election) I0507 20:09:30.104902 1276 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:30.105340 1276 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:30.105504 1276 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:30.105787 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.783s (SENT) I0507 20:09:30.105885 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:30.106098 1276 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.688s (election complete) I0507 20:09:30.106144 1276 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:30.106335 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:30.328156 1276 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:30.328272 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.232s (starting election) I0507 20:09:30.328349 1276 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:30.328783 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:30.328943 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:30.329675 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:30.343271 1276 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:30.343366 1276 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.365s (starting election) I0507 20:09:30.343435 1276 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:30.343804 1276 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:30.343937 1276 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:30.344213 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.404s (SENT) I0507 20:09:30.344374 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:30.344597 1276 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.317s (election complete) W0507 20:09:30.344630 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:30.344650 1276 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:30.561481 1276 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:30.561601 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.231s (starting election) I0507 20:09:30.561677 1276 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:30.562028 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.232s (SENT) I0507 20:09:30.562108 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:30.562108 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:30.562301 1276 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:30.562384 1276 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.380s (election complete) I0507 20:09:30.562412 1276 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:30.563043 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:30.662592 1280 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:30.662711 1280 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.156s (starting election) I0507 20:09:30.662806 1280 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:30.663290 1280 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:30.663465 1280 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:30.664427 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:30.709420 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.365s (SENT) I0507 20:09:30.709548 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:30.709823 1280 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (election complete) I0507 20:09:30.709897 1280 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:30.731580 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.624s (SENT) I0507 20:09:30.731660 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:30.731902 1280 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.836s (election complete) I0507 20:09:30.731966 1280 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:30.794445 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.231s (SENT) I0507 20:09:30.794543 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:30.794814 1280 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.543s (election complete) I0507 20:09:30.794901 1280 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:30.820214 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.156s (SENT) I0507 20:09:30.820317 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:30.820611 1280 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.747s (election complete) I0507 20:09:30.820683 1280 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:31.338763 1281 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:31.338866 1281 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.742s (starting election) I0507 20:09:31.338970 1281 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:31.339387 1281 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:31.339601 1281 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:31.340581 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:31.568666 1281 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:31.568773 1281 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.254s (starting election) I0507 20:09:31.568848 1281 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:31.569310 1281 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:31.569456 1281 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:31.569486 1283 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:31.569610 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.159s (starting election) I0507 20:09:31.569706 1283 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:31.570243 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:31.570289 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:31.570371 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:31.570838 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:31.730063 1283 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:31.730103 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.159s (SENT) I0507 20:09:31.730170 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.889s (starting election) I0507 20:09:31.730186 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:31.730278 1283 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:31.730690 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:31.730727 1281 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.380s (election complete) I0507 20:09:31.730818 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:31.730813 1281 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:31.731709 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:31.823518 1283 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:31.823648 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.311s (starting election) W0507 20:09:31.823829 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.254s (SENT) I0507 20:09:31.823740 1283 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:31.823930 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:31.824285 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:31.824306 1281 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.190s (election complete) I0507 20:09:31.824379 1281 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:31.825018 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:31.825426 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:32.015218 1283 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:32.015323 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.945s (starting election) I0507 20:09:32.015434 1283 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:32.015839 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:32.016000 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:32.016813 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:32.081624 1283 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:32.081735 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.434s (starting election) I0507 20:09:32.081820 1283 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:32.082267 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:32.082346 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.742s (SENT) I0507 20:09:32.082453 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:32.082460 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.082744 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.852s (election complete) I0507 20:09:32.082789 1283 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:32.083200 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:32.111340 1283 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:32.111507 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.809s (starting election) I0507 20:09:32.111625 1283 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:32.112249 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:32.112470 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:32.113284 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:32.119823 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:29.120323 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:29.120602 (+ 279us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:29.120618 (+ 16us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:29.120735 (+ 117us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:32.119368 (+2998633us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":82} W0507 20:09:32.120187 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:09:32.136759 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.311s (SENT) I0507 20:09:32.136862 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.137226 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.633s (election complete) I0507 20:09:32.137387 1283 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:32.518025 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.434s (SENT) I0507 20:09:32.518126 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.518461 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.121s (election complete) I0507 20:09:32.518540 1283 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:32.621515 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.889s (SENT) I0507 20:09:32.621651 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.622037 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.254s (election complete) I0507 20:09:32.622128 1283 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:32.640388 1283 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:32.640514 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.096s (starting election) I0507 20:09:32.640600 1283 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:32.641067 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:32.641245 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:32.642379 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:32.737092 1283 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:32.737200 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.428s (starting election) I0507 20:09:32.737283 1283 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:32.737706 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:32.737861 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:32.738132 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.096s (SENT) I0507 20:09:32.738188 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.738348 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.375s (election complete) I0507 20:09:32.738384 1283 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:32.738519 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:32.771340 1283 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:32.771427 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.102s (starting election) I0507 20:09:32.771498 1283 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:32.771916 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:32.772053 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:32.772850 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:32.874080 1283 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:32.874171 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.118s (starting election) I0507 20:09:32.874234 1283 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:32.874631 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:32.874763 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:32.875028 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.102s (SENT) I0507 20:09:32.875077 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.875305 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.909s (election complete) I0507 20:09:32.875414 1283 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:32.875461 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:32.876555 1283 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:32.876607 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.126s (starting election) I0507 20:09:32.876689 1283 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:32.877220 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:32.877408 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:32.878216 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:32.923334 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.809s (SENT) I0507 20:09:32.923445 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.923868 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.967s (election complete) I0507 20:09:32.923992 1283 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:32.962518 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.945s (SENT) I0507 20:09:32.962617 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.963043 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.981s (election complete) I0507 20:09:32.963177 1283 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:32.993716 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.118s (SENT) I0507 20:09:32.993813 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:32.994225 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.281s (election complete) I0507 20:09:32.994352 1283 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:33.004513 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.126s (SENT) I0507 20:09:33.004639 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:33.005043 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.386s (election complete) I0507 20:09:33.005177 1283 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:33.113899 1283 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:33.114049 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.691s (starting election) I0507 20:09:33.114156 1283 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:33.114709 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:33.114959 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:33.115983 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:33.166476 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.428s (SENT) I0507 20:09:33.166570 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:33.166901 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.295s (election complete) I0507 20:09:33.166971 1283 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:33.276147 1283 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:33.276293 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.294s (starting election) I0507 20:09:33.276399 1283 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:33.277014 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:33.277226 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:33.278131 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:33.392029 1283 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:33.392187 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (starting election) I0507 20:09:33.392292 1283 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:33.392879 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:33.393101 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:33.394173 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:33.462931 1283 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:33.463049 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.159s (starting election) I0507 20:09:33.463129 1283 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:33.463594 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:33.463766 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:33.464725 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:33.571729 1283 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:33.571887 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.226s (starting election) I0507 20:09:33.572005 1283 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:33.572319 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.294s (SENT) I0507 20:09:33.572398 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:33.572705 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:33.572880 1287 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.419s (election complete) I0507 20:09:33.572955 1283 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:33.572958 1287 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:33.573755 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:33.623167 1283 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:33.623276 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.144s (starting election) W0507 20:09:33.623464 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.159s (SENT) I0507 20:09:33.623354 1283 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:33.623541 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:33.623817 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:33.623831 1287 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.636s (election complete) I0507 20:09:33.623877 1287 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:33.623960 1283 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:33.624577 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:33.714386 1283 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:33.714440 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.321s (SENT) I0507 20:09:33.714567 1283 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.521s (starting election) I0507 20:09:33.714632 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:33.714697 1283 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:33.715303 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:33.715322 1287 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.092s (election complete) I0507 20:09:33.715364 1287 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:33.715503 1283 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:33.716343 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:33.768229 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.144s (SENT) I0507 20:09:33.768321 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:33.768676 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.749s (election complete) I0507 20:09:33.768800 1283 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:33.800221 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.226s (SENT) I0507 20:09:33.800343 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:33.800737 1283 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.775s (election complete) I0507 20:09:33.800853 1283 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:33.808228 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.691s (SENT) I0507 20:09:33.808354 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:33.808832 1283 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.446s (election complete) I0507 20:09:33.808954 1283 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:33.808987 1287 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:33.809056 1287 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.864s (starting election) I0507 20:09:33.809120 1287 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:33.809655 1287 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:33.809844 1287 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:33.810809 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:34.238160 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.521s (SENT) I0507 20:09:34.238267 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:34.238545 1287 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.596s (election complete) I0507 20:09:34.238605 1287 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:34.256109 1287 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:34.256230 1287 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.183s (starting election) I0507 20:09:34.256317 1287 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:34.256742 1287 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:34.256947 1287 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:34.257768 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:34.439918 1287 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:34.440030 1287 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.941s (starting election) I0507 20:09:34.440112 1287 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:34.440560 1287 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:34.440685 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.183s (SENT) I0507 20:09:34.440723 1287 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:34.440745 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:34.440963 1287 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.149s (election complete) I0507 20:09:34.441016 1287 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:34.441722 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:34.577297 1287 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:34.577419 1287 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.206s (starting election) I0507 20:09:34.577497 1287 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:34.577935 1287 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:34.578104 1287 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:34.579130 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:34.591074 1287 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:34.591172 1287 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.087s (starting election) I0507 20:09:34.591246 1287 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:34.591691 1287 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:34.591843 1287 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:34.592866 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:34.675081 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.864s (SENT) I0507 20:09:34.675213 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:34.675599 1287 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.802s (election complete) I0507 20:09:34.675742 1287 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:34.678902 1287 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:34.678982 1287 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.373s (starting election) I0507 20:09:34.679054 1287 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:34.679265 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.087s (SENT) I0507 20:09:34.679345 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:34.679455 1287 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:34.679630 1287 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:34.679723 1287 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.387s (election complete) I0507 20:09:34.679754 1287 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:34.680603 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:34.784364 1288 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:34.784471 1288 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.125s (starting election) I0507 20:09:34.784570 1288 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:34.785007 1288 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:34.785125 1288 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:34.785360 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.206s (SENT) I0507 20:09:34.785421 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:09:34.785715 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:34.785775 1288 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.295s (election complete) I0507 20:09:34.785809 1288 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:34.911064 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.125s (SENT) I0507 20:09:34.911175 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:34.911504 1288 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.571s (election complete) I0507 20:09:34.911582 1288 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:35.053639 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.373s (SENT) I0507 20:09:35.053761 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:35.054097 1288 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.539s (election complete) I0507 20:09:35.054172 1288 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:35.383165 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.941s (SENT) I0507 20:09:35.383258 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:35.383553 1288 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.473s (election complete) I0507 20:09:35.383630 1288 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:35.478576 1288 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:35.478695 1288 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.562s (starting election) I0507 20:09:35.478793 1288 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:35.479234 1288 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:35.479394 1288 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:35.481197 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:35.483255 1288 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:35.483330 1288 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.623s (starting election) I0507 20:09:35.483388 1288 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:35.483738 1288 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:35.483865 1288 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:35.484632 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:35.771551 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:32.771758 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:32.772172 (+ 414us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:32.772189 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:32.772319 (+ 130us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:35.771236 (+2998917us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":191} W0507 20:09:35.771857 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:35.857201 1288 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:35.857336 1288 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.269s (starting election) I0507 20:09:35.857477 1288 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:35.858023 1288 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:35.858235 1288 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:35.859160 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:36.042045 1288 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:36.042165 1288 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.404s (starting election) I0507 20:09:36.042271 1288 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:36.042814 1288 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:36.043015 1288 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:36.043370 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.562s (SENT) I0507 20:09:36.043464 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:36.043694 1288 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.696s (election complete) I0507 20:09:36.043781 1288 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:36.043992 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:36.107664 1288 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:36.107810 1288 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.372s (starting election) I0507 20:09:36.107923 1288 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:36.108199 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.623s (SENT) I0507 20:09:36.108314 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:36.108431 1288 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:36.108656 1288 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:36.108804 1288 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.448s (election complete) I0507 20:09:36.108906 1288 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:36.109799 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:36.127189 1288 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:36.127295 1288 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.978s (starting election) I0507 20:09:36.127373 1288 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:36.127563 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.269s (SENT) I0507 20:09:36.127662 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:36.127807 1288 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:36.127920 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.860s (election complete) I0507 20:09:36.127962 1288 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:36.128000 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:36.128610 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:36.447458 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.404s (SENT) I0507 20:09:36.447597 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:36.447921 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.270s (election complete) I0507 20:09:36.448052 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:36.482100 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.372s (SENT) I0507 20:09:36.482208 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:36.482555 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.184s (election complete) I0507 20:09:36.482687 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:36.667750 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:36.667913 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.488s (starting election) I0507 20:09:36.668018 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:36.668522 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:36.668730 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:36.669859 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:36.719457 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:36.719621 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.552s (starting election) I0507 20:09:36.719722 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:36.720260 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:36.720468 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:36.721319 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:36.989184 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:36.989307 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.141s (starting election) I0507 20:09:36.989423 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:36.989925 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:36.990126 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:36.990907 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:37.107914 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.978s (SENT) I0507 20:09:37.108067 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:37.108490 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.545s (election complete) I0507 20:09:37.108628 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:37.131861 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.141s (SENT) I0507 20:09:37.132017 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:37.132391 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.356s (election complete) I0507 20:09:37.132517 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:37.156628 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:37.156781 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.080s (starting election) I0507 20:09:37.156903 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:37.157459 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:37.157650 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.488s (SENT) I0507 20:09:37.157704 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:37.157804 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:37.158052 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.105s (election complete) I0507 20:09:37.158124 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:37.158409 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:37.238430 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.080s (SENT) I0507 20:09:37.238597 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:37.238979 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.340s (election complete) I0507 20:09:37.239089 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:37.272723 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:37.272897 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.240s (starting election) I0507 20:09:37.273038 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:37.273632 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:37.273814 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.552s (SENT) I0507 20:09:37.273886 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:37.273938 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:37.274166 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.993s (election complete) I0507 20:09:37.274240 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:37.274797 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:37.489617 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:37.489739 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (starting election) I0507 20:09:37.489857 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:37.490329 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:37.490481 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:37.491375 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:37.514480 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.240s (SENT) I0507 20:09:37.514597 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:37.514976 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.160s (election complete) I0507 20:09:37.515091 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:37.579943 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:37.580060 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.449s (starting election) I0507 20:09:37.580143 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:37.580600 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:37.580765 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:37.581610 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:37.676151 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:37.676271 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.128s (starting election) I0507 20:09:37.676373 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:37.676834 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:37.677008 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:37.678014 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:37.804646 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:37.804774 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.740s (starting election) I0507 20:09:37.804852 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:37.805303 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:37.805495 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:37.806051 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.128s (SENT) I0507 20:09:37.806102 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:37.806267 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.927s (election complete) I0507 20:09:37.806315 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:37.806417 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:38.030318 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:38.030450 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.321s (starting election) I0507 20:09:38.030546 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:38.030961 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.449s (SENT) I0507 20:09:38.031052 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:38.031102 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:38.031375 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:38.031570 1294 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.389s (election complete) I0507 20:09:38.031626 1294 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:38.032048 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:38.353703 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.321s (SENT) I0507 20:09:38.353806 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:38.354095 1294 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.892s (election complete) I0507 20:09:38.354084 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:38.354169 1294 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:38.354195 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.280s (starting election) I0507 20:09:38.354293 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:38.354746 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:38.354950 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:38.355227 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.863s (SENT) I0507 20:09:38.355273 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:38.355484 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.267s (election complete) I0507 20:09:38.355545 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:38.355648 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:38.546771 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.740s (SENT) I0507 20:09:38.546900 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:38.547250 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.724s (election complete) I0507 20:09:38.547354 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:38.623208 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:38.623301 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.605s (starting election) I0507 20:09:38.623369 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:38.623764 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:38.623889 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:38.624719 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:38.635761 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.280s (SENT) I0507 20:09:38.635874 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:38.636242 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.244s (election complete) I0507 20:09:38.636335 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:38.881307 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:38.881424 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.905s (starting election) I0507 20:09:38.881518 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:38.881947 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:38.882104 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:38.882966 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:39.229539 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.605s (SENT) I0507 20:09:39.229635 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:39.229921 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.478s (election complete) I0507 20:09:39.229991 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:39.246490 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:39.246630 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.955s (starting election) I0507 20:09:39.246717 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:39.247145 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:39.247326 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:39.247990 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:39.272439 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:39.272547 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.203s (starting election) I0507 20:09:39.272625 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:39.273067 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:39.273219 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:39.274140 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:39.431586 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:36.431148 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:36.431460 (+ 312us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:36.431476 (+ 16us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:36.431607 (+ 131us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:39.431221 (+2999614us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":90} W0507 20:09:39.431926 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:39.475953 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:39.476074 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.475s (starting election) I0507 20:09:39.476150 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:39.476603 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:39.476769 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:39.477008 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.203s (SENT) I0507 20:09:39.477111 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:39.477289 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.990s (election complete) I0507 20:09:39.477322 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:39.477743 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:39.708691 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:39.708804 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.194s (starting election) I0507 20:09:39.708883 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:39.709331 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:39.709506 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:39.710331 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:39.787813 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.905s (SENT) I0507 20:09:39.787905 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:39.788156 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.881s (election complete) I0507 20:09:39.788239 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:39.903803 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.194s (SENT) I0507 20:09:39.903913 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:39.904227 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.546s (election complete) I0507 20:09:39.904328 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:39.952955 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.475s (SENT) I0507 20:09:39.953070 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:39.953450 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.672s (election complete) I0507 20:09:39.953557 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:40.202047 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:40.202164 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.603s (starting election) I0507 20:09:40.202245 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:40.202733 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:40.202939 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:40.203785 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:40.204278 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.955s (SENT) I0507 20:09:40.204331 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:40.204571 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.250s (election complete) I0507 20:09:40.204638 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:40.450743 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:40.450891 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.651s (starting election) I0507 20:09:40.450984 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:40.451503 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:40.451742 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:40.452595 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:40.455019 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:40.455096 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.806s (starting election) I0507 20:09:40.455181 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:40.455581 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:40.455705 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:40.456493 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:40.626780 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:40.626920 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.919s (starting election) I0507 20:09:40.627004 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:40.627429 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:40.627606 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:40.628438 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:40.807749 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.603s (SENT) I0507 20:09:40.807862 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:40.808214 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.691s (election complete) I0507 20:09:40.808326 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:41.102694 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:41.102830 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.807s (starting election) I0507 20:09:41.102946 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:41.103423 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:41.103641 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:41.103969 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.651s (SENT) I0507 20:09:41.104053 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:41.104234 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.574s (election complete) I0507 20:09:41.104279 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:41.104501 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:41.263347 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.806s (SENT) I0507 20:09:41.263458 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:41.263792 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.850s (election complete) I0507 20:09:41.263885 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:41.547072 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:41.547111 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.919s (SENT) I0507 20:09:41.547212 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.447s (starting election) I0507 20:09:41.547252 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:41.547307 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:41.547829 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:41.547844 1297 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.804s (election complete) I0507 20:09:41.547907 1297 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:41.547999 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:41.548722 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:41.679247 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:41.679355 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.112s (starting election) I0507 20:09:41.679428 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:41.679833 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:41.679989 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:41.680708 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:41.792151 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:41.792243 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.724s (starting election) W0507 20:09:41.792263 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.112s (SENT) I0507 20:09:41.792345 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:41.792305 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:41.792676 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:41.792685 1297 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.191s (election complete) I0507 20:09:41.792727 1297 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:41.792793 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:41.793642 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:41.911104 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.807s (SENT) I0507 20:09:41.911221 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:41.911569 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.267s (election complete) I0507 20:09:41.911669 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:41.995479 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.447s (SENT) I0507 20:09:41.995575 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:41.995925 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.464s (election complete) I0507 20:09:41.996028 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:42.114706 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:42.114843 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.828s (starting election) I0507 20:09:42.114962 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:42.115469 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:42.115649 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:42.116530 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:42.179632 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:42.179749 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (starting election) I0507 20:09:42.179834 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:42.180284 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:42.180447 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:42.181162 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:42.460566 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:42.460674 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.588s (starting election) I0507 20:09:42.460749 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:42.461252 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:42.461427 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:42.462404 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:42.518088 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.724s (SENT) I0507 20:09:42.518193 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:42.518509 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.438s (election complete) I0507 20:09:42.518622 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:42.898613 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.716s (SENT) I0507 20:09:42.898710 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:42.899024 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.375s (election complete) I0507 20:09:42.899097 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:42.943572 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:42.943688 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.688s (starting election) I0507 20:09:42.943768 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:42.944213 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:42.944378 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:42.945274 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:42.945950 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.828s (SENT) I0507 20:09:42.946015 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:42.946221 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.217s (election complete) I0507 20:09:42.946286 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:43.049856 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:43.049971 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.149s (starting election) I0507 20:09:43.050055 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:43.050493 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:43.050645 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:43.050922 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.588s (SENT) I0507 20:09:43.050998 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:43.051195 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.327s (election complete) I0507 20:09:43.051254 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:43.051651 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:43.103483 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:40.104667 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:40.104991 (+ 324us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:40.105008 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:40.105135 (+ 127us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:43.103193 (+2998058us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":95} W0507 20:09:43.103790 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:43.164419 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:43.164547 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.391s (starting election) I0507 20:09:43.164635 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:43.165204 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:43.165402 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:43.166329 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:43.200023 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.149s (SENT) I0507 20:09:43.200117 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:43.200394 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.413s (election complete) I0507 20:09:43.200461 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:43.274864 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:43.275028 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.725s (starting election) I0507 20:09:43.275126 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:43.275673 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:43.275866 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:43.276623 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:43.556728 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:43.556864 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.557s (starting election) W0507 20:09:43.556939 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.391s (SENT) I0507 20:09:43.557024 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:43.556962 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:43.557512 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:43.557574 1299 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.762s (election complete) I0507 20:09:43.557641 1299 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:43.557701 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:43.558552 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:43.614182 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:43.614322 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.905s (starting election) I0507 20:09:43.614429 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:43.614938 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:43.615137 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:43.616086 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:43.633509 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.688s (SENT) I0507 20:09:43.633623 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:43.633980 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.757s (election complete) I0507 20:09:43.634100 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:44.001505 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:44.001639 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.562s (starting election) I0507 20:09:44.001742 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:44.002411 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:44.002661 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:44.002774 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.725s (SENT) I0507 20:09:44.002852 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:44.003103 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.750s (election complete) I0507 20:09:44.003168 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:44.003445 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:44.116390 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.557s (SENT) I0507 20:09:44.116492 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:44.116919 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.574s (election complete) I0507 20:09:44.117085 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:44.519664 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:44.519776 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.693s (starting election) I0507 20:09:44.519853 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:44.520278 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:44.520426 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:44.520691 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.905s (SENT) I0507 20:09:44.520773 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:44.520989 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.995s (election complete) I0507 20:09:44.521045 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:44.521384 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:44.566305 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.562s (SENT) I0507 20:09:44.566407 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:44.566664 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.858s (election complete) I0507 20:09:44.566735 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:44.692417 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:44.692530 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.150s (starting election) I0507 20:09:44.692611 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:44.693083 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:44.693245 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:44.694026 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:44.843490 1291 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:44.843597 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.868s (starting election) I0507 20:09:44.843691 1291 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:44.844173 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:44.844300 1291 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:44.844347 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.150s (SENT) I0507 20:09:44.844404 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:44.844574 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.906s (election complete) I0507 20:09:44.844633 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:44.845084 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:45.214200 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.693s (SENT) I0507 20:09:45.214294 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:45.214583 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.445s (election complete) I0507 20:09:45.214679 1291 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:45.425678 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:45.425781 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.471s (starting election) I0507 20:09:45.425875 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:45.426306 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:45.426465 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:45.428061 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:45.660842 1291 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:45.660979 1291 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.994s (starting election) I0507 20:09:45.661075 1291 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:45.661520 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:45.661691 1291 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:45.662541 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:45.714920 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.868s (SENT) I0507 20:09:45.715056 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:45.715556 1291 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.184s (election complete) I0507 20:09:45.715656 1291 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:45.898103 1291 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:45.898206 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.226s (starting election) I0507 20:09:45.898277 1291 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:45.898530 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.471s (SENT) I0507 20:09:45.898649 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:45.898691 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:45.898847 1291 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:45.898998 1291 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.574s (election complete) I0507 20:09:45.899034 1291 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:45.899629 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:45.899821 1303 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:45.899868 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.748s (starting election) I0507 20:09:45.899909 1303 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:45.900199 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:45.900291 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:45.900934 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:46.126140 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.226s (SENT) I0507 20:09:46.126235 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:46.126518 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.083s (election complete) I0507 20:09:46.126585 1303 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:46.210273 1303 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:46.210381 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.227s (starting election) I0507 20:09:46.210454 1303 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:46.210865 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:46.211040 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:46.212085 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:46.438735 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.227s (SENT) I0507 20:09:46.438747 1303 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:46.438856 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:46.438860 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.634s (starting election) I0507 20:09:46.439024 1303 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:46.439507 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:46.439519 1304 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.498s (election complete) I0507 20:09:46.439599 1304 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:46.439700 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:46.440340 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:46.648480 1303 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:46.648615 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.615s (starting election) I0507 20:09:46.648756 1303 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:46.649224 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:46.649369 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:46.649698 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.748s (SENT) I0507 20:09:46.649827 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:46.650260 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.859s (election complete) W0507 20:09:46.650262 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:46.650359 1303 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:46.656003 1303 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:46.656165 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.221s (starting election) I0507 20:09:46.656265 1303 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:46.656735 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:46.656904 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:46.657153 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.994s (SENT) I0507 20:09:46.657238 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:46.657464 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.468s (election complete) I0507 20:09:46.657582 1303 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:46.657858 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:46.783548 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:43.782714 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:43.782986 (+ 272us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:43.783001 (+ 15us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:43.783097 (+ 96us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:46.783268 (+3000171us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":79} W0507 20:09:46.783849 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:09:46.879184 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.221s (SENT) I0507 20:09:46.879289 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:46.879611 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.852s (election complete) I0507 20:09:46.879714 1303 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:46.938644 1303 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:46.938761 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.933s (starting election) I0507 20:09:46.938861 1303 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:46.939319 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:46.939503 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:46.940352 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:47.075345 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.634s (SENT) I0507 20:09:47.075469 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:47.075844 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.504s (election complete) I0507 20:09:47.075963 1303 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:47.266274 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.615s (SENT) I0507 20:09:47.266391 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:47.266738 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.930s (election complete) I0507 20:09:47.266826 1303 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:47.580446 1303 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:47.580572 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.995s (starting election) I0507 20:09:47.580662 1303 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:47.581158 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:47.581346 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:47.582389 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:47.732746 1303 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:47.732888 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.287s (starting election) I0507 20:09:47.732993 1303 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:47.733438 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:47.733613 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:47.734385 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:47.873492 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.933s (SENT) I0507 20:09:47.873605 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:47.873966 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.473s (election complete) I0507 20:09:47.874047 1303 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:48.020754 1303 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:48.020817 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.287s (SENT) I0507 20:09:48.020869 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.093s (starting election) I0507 20:09:48.020895 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:48.020962 1303 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:48.021401 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:48.021404 1308 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.913s (election complete) I0507 20:09:48.021457 1308 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:48.021579 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:48.022451 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:48.114826 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.093s (SENT) I0507 20:09:48.114953 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:48.115273 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.619s (election complete) I0507 20:09:48.115350 1303 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:48.198164 1303 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:48.198281 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.125s (starting election) I0507 20:09:48.198385 1303 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:48.198842 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:48.199030 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:48.199798 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:48.324497 1303 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:48.324620 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.573s (starting election) W0507 20:09:48.324693 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.125s (SENT) I0507 20:09:48.324779 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:48.324714 1303 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:48.325196 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:48.325232 1308 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.642s (election complete) I0507 20:09:48.325296 1308 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:48.325377 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:48.326088 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:48.347614 1303 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:48.347733 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.919s (starting election) I0507 20:09:48.347831 1303 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:48.348278 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:48.348472 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:48.349236 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:48.577826 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.995s (SENT) I0507 20:09:48.577944 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:48.578248 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.217s (election complete) I0507 20:09:48.578335 1303 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:48.735436 1303 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:48.735570 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.286s (starting election) I0507 20:09:48.735688 1303 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:48.736258 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:48.736480 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:48.737347 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:48.795838 1303 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:48.795949 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.169s (starting election) I0507 20:09:48.796051 1303 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:48.796504 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:48.796649 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:48.797319 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:48.899294 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.573s (SENT) I0507 20:09:48.899408 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:48.899751 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.085s (election complete) I0507 20:09:48.899833 1303 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:48.966284 1303 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:48.966310 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.169s (SENT) I0507 20:09:48.966401 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.099s (starting election) I0507 20:09:48.966434 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:48.966488 1303 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:48.966936 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:48.966939 1309 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.502s (election complete) I0507 20:09:48.966996 1309 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:48.967097 1303 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:48.967819 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:48.985347 1303 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:48.985472 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.708s (starting election) I0507 20:09:48.985564 1303 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:48.986003 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:48.986182 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:48.987061 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:49.022428 1303 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:49.022542 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.522s (starting election) I0507 20:09:49.022647 1303 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:49.023157 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:49.023341 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:49.023579 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.286s (SENT) I0507 20:09:49.023663 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:49.023859 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.218s (election complete) I0507 20:09:49.023936 1303 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:49.024101 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:49.066812 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.099s (SENT) I0507 20:09:49.066928 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:49.067260 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.236s (election complete) I0507 20:09:49.067382 1303 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:49.243283 1303 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:49.243422 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.351s (starting election) I0507 20:09:49.243520 1303 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:49.244004 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:49.244166 1303 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:49.245123 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:49.268689 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.919s (SENT) I0507 20:09:49.268790 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:49.269045 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.624s (election complete) I0507 20:09:49.269129 1303 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:49.547250 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.522s (SENT) I0507 20:09:49.547353 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:49.547663 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.320s (election complete) I0507 20:09:49.547744 1303 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:49.596688 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.351s (SENT) I0507 20:09:49.596802 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:49.597151 1303 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.364s (election complete) I0507 20:09:49.597242 1303 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:49.694788 1303 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:49.694923 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.830s (starting election) I0507 20:09:49.695020 1303 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:49.695483 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.708s (SENT) I0507 20:09:49.695538 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:49.695567 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:49.695730 1303 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:49.695848 1303 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.554s (election complete) I0507 20:09:49.695894 1303 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:49.696507 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:49.894002 1310 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:49.894122 1310 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.148s (starting election) I0507 20:09:49.894212 1310 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:49.894708 1310 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:49.894918 1310 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:49.895648 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:49.962162 1310 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:49.962280 1310 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.595s (starting election) I0507 20:09:49.962378 1310 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:49.962837 1310 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:49.963028 1310 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:49.963918 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:50.043550 1310 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:50.043613 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.148s (SENT) I0507 20:09:50.043663 1310 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.180s (starting election) I0507 20:09:50.043709 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:50.043730 1310 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:50.044160 1310 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:50.044200 1303 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.543s (election complete) I0507 20:09:50.044257 1303 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:50.044317 1310 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:50.045028 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:50.225037 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.180s (SENT) I0507 20:09:50.225148 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:50.225473 1310 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.720s (election complete) I0507 20:09:50.225548 1310 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:50.250644 1310 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:50.250762 1310 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.529s (starting election) I0507 20:09:50.250867 1310 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:50.251392 1310 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:50.251574 1310 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:50.252312 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:50.467485 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:47.469405 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:47.469658 (+ 253us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:47.469671 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:47.469830 (+ 159us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:50.467179 (+2997349us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":90} W0507 20:09:50.467761 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:09:50.527879 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.830s (SENT) I0507 20:09:50.527987 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:50.528290 1310 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.441s (election complete) I0507 20:09:50.528357 1310 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:50.558367 1310 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:50.558472 1310 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.220s (starting election) I0507 20:09:50.558545 1310 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:50.559022 1310 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:50.559185 1310 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:50.559378 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.595s (SENT) I0507 20:09:50.559434 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:50.559640 1310 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.298s (election complete) I0507 20:09:50.559695 1310 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:50.559921 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:50.779778 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.220s (SENT) I0507 20:09:50.779878 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:50.780164 1310 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.978s (election complete) I0507 20:09:50.780246 1310 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:50.782438 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.529s (SENT) I0507 20:09:50.782510 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:50.782728 1310 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.947s (election complete) I0507 20:09:50.782802 1310 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:50.946525 1310 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:50.946646 1310 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.416s (starting election) I0507 20:09:50.946722 1310 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:50.947154 1310 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:50.947304 1310 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:50.948034 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:51.363660 1310 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:51.363775 1310 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.855s (starting election) I0507 20:09:51.363864 1310 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:51.364187 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.416s (SENT) I0507 20:09:51.364251 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:51.364317 1310 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:51.364440 1310 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:51.364534 1310 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.209s (election complete) I0507 20:09:51.364578 1310 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:51.365276 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:51.574285 1313 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:51.574396 1313 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.531s (starting election) I0507 20:09:51.574468 1313 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:51.574903 1313 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:51.575084 1313 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:51.575716 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:51.730108 1313 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:51.730207 1313 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.538s (starting election) I0507 20:09:51.730284 1313 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:51.730672 1313 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:51.730813 1313 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:51.731511 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:51.758764 1313 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:51.758864 1313 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.774s (starting election) I0507 20:09:51.758954 1313 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:51.759366 1313 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:51.759512 1313 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:51.760205 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:52.105805 1313 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:52.105926 1313 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.849s (starting election) I0507 20:09:52.106029 1313 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:52.106465 1313 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:52.106601 1313 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:52.107172 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:52.108068 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.531s (SENT) I0507 20:09:52.108130 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:52.108299 1313 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.856s (election complete) I0507 20:09:52.108366 1313 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:52.220790 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.855s (SENT) I0507 20:09:52.220885 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:52.221145 1313 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.407s (election complete) I0507 20:09:52.221238 1313 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:52.269196 1313 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:52.269297 1313 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.797s (starting election) I0507 20:09:52.269376 1313 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:52.269784 1313 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:52.269930 1313 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:52.270398 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.538s (SENT) I0507 20:09:52.270503 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:09:52.270606 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:52.270802 1313 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.918s (election complete) I0507 20:09:52.270860 1313 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:52.534245 1313 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:52.534302 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.774s (SENT) I0507 20:09:52.534354 1313 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.100s (starting election) I0507 20:09:52.534384 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:52.534438 1313 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:52.534906 1313 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:52.535087 1313 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:52.535142 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.402s (election complete) I0507 20:09:52.535235 1314 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:52.535985 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:52.629195 1314 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:52.629348 1314 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.949s (starting election) I0507 20:09:52.629580 1314 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:52.630187 1314 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:52.630399 1314 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:52.631086 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:52.635496 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.100s (SENT) I0507 20:09:52.635581 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:52.635846 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.453s (election complete) I0507 20:09:52.635926 1314 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:52.957310 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.849s (SENT) I0507 20:09:52.957412 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:52.957773 1314 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.271s (election complete) I0507 20:09:52.957859 1314 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:53.068436 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.797s (SENT) I0507 20:09:53.068542 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:53.068871 1314 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.432s (election complete) I0507 20:09:53.068965 1314 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:53.090250 1314 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:53.090358 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.249s (starting election) I0507 20:09:53.090443 1314 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:53.090894 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:53.091069 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:53.091763 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:53.230257 1314 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:53.230401 1314 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.191s (starting election) I0507 20:09:53.230499 1314 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:53.231065 1314 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:53.231273 1314 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:53.232048 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:53.340373 1314 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:53.340414 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.249s (SENT) I0507 20:09:53.340530 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.213s (starting election) I0507 20:09:53.340554 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:53.340663 1314 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:53.341166 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:53.341359 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:53.341504 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.103s (election complete) I0507 20:09:53.341598 1314 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:53.342226 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:53.422459 1315 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:53.422650 1315 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.874s (starting election) I0507 20:09:53.422770 1315 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:53.423298 1315 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:53.423398 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.191s (SENT) I0507 20:09:53.423498 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:53.423547 1315 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:53.423755 1314 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.417s (election complete) I0507 20:09:53.423836 1314 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:53.424289 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:53.444913 1314 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:53.445026 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.859s (starting election) I0507 20:09:53.445116 1314 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:53.445612 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:53.445797 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:53.446578 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:53.502328 1314 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:53.502471 1314 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.318s (starting election) I0507 20:09:53.502565 1314 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:53.503015 1314 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:53.503186 1314 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:53.503926 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:53.555907 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.213s (SENT) I0507 20:09:53.556004 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:53.556309 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.075s (election complete) I0507 20:09:53.556416 1314 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:53.581532 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.949s (SENT) I0507 20:09:53.581647 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:53.581956 1314 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.451s (election complete) I0507 20:09:53.582054 1314 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:53.632035 1314 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:53.632153 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.457s (starting election) I0507 20:09:53.632241 1314 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:53.632658 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:53.632804 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:53.633848 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:53.821799 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.318s (SENT) I0507 20:09:53.821758 1314 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:53.821916 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:53.821957 1314 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.861s (starting election) I0507 20:09:53.822088 1314 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:53.822690 1314 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:53.822736 1315 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.943s (election complete) I0507 20:09:53.822829 1315 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:53.822938 1314 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:53.823714 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:54.034225 1314 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:54.034397 1314 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.336s (starting election) I0507 20:09:54.034533 1314 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:54.035270 1314 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:54.035528 1314 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:54.036411 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:54.090502 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.457s (SENT) I0507 20:09:54.090548 1314 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:54.090725 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.110s (starting election) I0507 20:09:54.090912 1314 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:54.091905 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:54.092118 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:54.090631 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:54.092746 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.242s (election complete) I0507 20:09:54.092823 1314 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:54.093106 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:54.159471 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:51.159233 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:51.159433 (+ 200us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:51.159446 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:51.159530 (+ 84us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:54.159148 (+2999618us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":55} W0507 20:09:54.159741 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:09:54.204262 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.110s (SENT) I0507 20:09:54.204396 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:54.204824 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.563s (election complete) I0507 20:09:54.204952 1314 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:54.299072 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.874s (SENT) I0507 20:09:54.299182 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:54.299536 1314 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.788s (election complete) I0507 20:09:54.299635 1314 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:54.306066 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.859s (SENT) I0507 20:09:54.306180 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:54.306499 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.195s (election complete) I0507 20:09:54.306608 1314 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:54.372200 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.336s (SENT) I0507 20:09:54.372336 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:54.372699 1314 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.879s (election complete) I0507 20:09:54.372802 1314 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:54.502549 1314 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:54.502666 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.261s (starting election) I0507 20:09:54.502763 1314 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:54.503235 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:54.503417 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:54.504176 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:54.685020 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.861s (SENT) I0507 20:09:54.685133 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:54.685501 1314 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.584s (election complete) I0507 20:09:54.685592 1314 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:54.764034 1314 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:54.764156 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.459s (starting election) I0507 20:09:54.764256 1314 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:54.764737 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:54.764937 1314 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:54.765130 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.261s (SENT) I0507 20:09:54.765203 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:09:54.765847 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:54.766057 1314 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.347s (election complete) I0507 20:09:54.766135 1314 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:55.114565 1317 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:55.114694 1317 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.209s (starting election) I0507 20:09:55.114794 1317 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:55.115274 1317 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:55.115484 1317 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:55.116255 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:55.224758 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.459s (SENT) I0507 20:09:55.224869 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:55.225185 1317 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.463s (election complete) I0507 20:09:55.225281 1317 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:55.253175 1317 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:55.253319 1317 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.174s (starting election) I0507 20:09:55.253419 1317 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:55.253872 1317 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:55.254058 1317 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:55.255029 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:55.270225 1317 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:55.270350 1317 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.089s (starting election) I0507 20:09:55.270439 1317 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:55.270912 1317 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:55.271087 1317 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:55.271957 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:55.324875 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.209s (SENT) I0507 20:09:55.324992 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:55.325321 1317 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.107s (election complete) I0507 20:09:55.325419 1317 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:55.360007 1317 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:55.360124 1317 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.507s (starting election) W0507 20:09:55.360359 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.089s (SENT) I0507 20:09:55.360265 1317 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:55.360437 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:55.360695 1317 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:55.360875 1317 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:55.361028 1317 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.485s (election complete) I0507 20:09:55.361099 1317 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:55.361788 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:55.428433 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.174s (SENT) I0507 20:09:55.428560 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:55.428807 1319 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:55.428918 1319 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.999s (starting election) I0507 20:09:55.429041 1319 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:55.429512 1319 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:55.429574 1317 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.645s (election complete) I0507 20:09:55.429709 1317 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:55.429765 1319 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:55.430552 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:55.432894 1319 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:55.432974 1319 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.768s (starting election) I0507 20:09:55.433111 1319 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:55.433483 1319 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:55.433616 1319 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:55.434348 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:55.846819 1319 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:55.846956 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.115s (starting election) I0507 20:09:55.847056 1319 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:55.847520 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:55.847672 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:55.848505 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:55.869427 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.507s (SENT) I0507 20:09:55.869540 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:55.869855 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.240s (election complete) I0507 20:09:55.869958 1319 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:55.963776 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.115s (SENT) I0507 20:09:55.963874 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:55.964176 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.524s (election complete) I0507 20:09:55.964260 1319 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:56.075371 1319 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:56.075484 1319 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.111s (starting election) I0507 20:09:56.075572 1319 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:56.076041 1319 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:56.076215 1319 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:56.077139 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:56.187088 1319 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:56.187203 1319 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.701s (starting election) I0507 20:09:56.187306 1319 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:56.187796 1319 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:56.187999 1319 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:56.188300 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.111s (SENT) I0507 20:09:56.188367 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:56.188534 1319 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.600s (election complete) I0507 20:09:56.188587 1319 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:56.188982 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:56.202383 1319 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:56.202448 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.768s (SENT) I0507 20:09:56.202503 1319 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.961s (starting election) I0507 20:09:56.202533 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:56.202589 1319 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:56.203254 1319 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:56.203266 1320 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.455s (election complete) I0507 20:09:56.203434 1319 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:56.203431 1320 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:56.204450 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:56.430536 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.999s (SENT) I0507 20:09:56.430645 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:56.430958 1319 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.711s (election complete) I0507 20:09:56.431047 1319 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:56.489562 1319 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:56.489665 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.122s (starting election) I0507 20:09:56.489748 1319 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:56.490173 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:56.490355 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:56.491364 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:56.613634 1319 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:56.613785 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.122s (SENT) I0507 20:09:56.613801 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.238s (starting election) I0507 20:09:56.613958 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:56.614043 1319 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:56.614583 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:56.614645 1320 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.126s (election complete) I0507 20:09:56.614758 1320 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:56.614806 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:56.615571 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:56.659644 1319 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:56.659782 1319 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.281s (starting election) I0507 20:09:56.659898 1319 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:56.660490 1319 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:56.660717 1319 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:56.661813 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:56.742130 1319 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:56.742270 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.706s (starting election) I0507 20:09:56.742389 1319 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:56.743016 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:56.743239 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:56.744331 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:56.853559 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.238s (SENT) I0507 20:09:56.853670 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:56.853969 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.998s (election complete) I0507 20:09:56.854041 1319 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:56.889860 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.701s (SENT) I0507 20:09:56.889987 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:56.890333 1319 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.894s (election complete) I0507 20:09:56.890431 1319 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:09:56.942162 1319 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:56.942240 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.281s (SENT) I0507 20:09:56.942289 1319 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.700s (starting election) I0507 20:09:56.942348 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:56.942404 1319 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:56.942903 1319 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:56.942908 1320 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.419s (election complete) I0507 20:09:56.942986 1320 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:56.943110 1319 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:56.943989 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:57.165568 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.961s (SENT) I0507 20:09:57.165696 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:57.166177 1319 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.571s (election complete) I0507 20:09:57.166292 1319 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:57.450361 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.706s (SENT) I0507 20:09:57.450464 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:57.450808 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.581s (election complete) I0507 20:09:57.450932 1319 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:57.644223 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.700s (SENT) I0507 20:09:57.644351 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:57.644701 1319 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.919s (election complete) I0507 20:09:57.644811 1319 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:57.785650 1319 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:57.785758 1319 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.945s (starting election) I0507 20:09:57.785857 1319 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:57.786322 1319 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:57.786471 1319 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:57.787904 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:57.863519 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:54.863036 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:54.863341 (+ 305us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:54.863358 (+ 17us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:54.863463 (+ 105us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:09:57.863235 (+2999772us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":88} W0507 20:09:57.863776 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:09:58.032985 1319 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:58.033098 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.180s (starting election) I0507 20:09:58.033187 1319 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:58.033586 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:58.033712 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:58.034441 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:58.214407 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.180s (SENT) I0507 20:09:58.214478 1319 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:58.214546 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:58.214582 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.089s (starting election) I0507 20:09:58.214679 1319 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:58.215195 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:58.215396 1319 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:58.215560 1319 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.845s (election complete) I0507 20:09:58.215622 1319 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:58.216259 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:58.305884 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.089s (SENT) I0507 20:09:58.305989 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:58.306325 1323 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.710s (election complete) I0507 20:09:58.306413 1323 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:58.565162 1323 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:58.565274 1323 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.601s (starting election) I0507 20:09:58.565421 1323 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:58.565912 1323 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:58.566125 1323 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:58.567268 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:58.732043 1323 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:09:58.732084 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.945s (SENT) I0507 20:09:58.732225 1323 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.748s (starting election) I0507 20:09:58.732275 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:58.732316 1323 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:58.732719 1323 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:58.732880 1323 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:58.733000 1323 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.576s (election complete) I0507 20:09:58.733072 1323 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:58.733613 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:59.017561 1325 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:59.017678 1325 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.682s (starting election) I0507 20:09:59.017760 1325 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:59.018218 1325 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:59.018576 1325 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:59.019440 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:59.167309 1325 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:59.167428 1325 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.387s (starting election) I0507 20:09:59.167512 1325 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:09:59.167920 1325 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:59.168099 1325 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:09:59.168308 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.601s (SENT) I0507 20:09:59.168388 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:59.168571 1325 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.457s (election complete) I0507 20:09:59.168674 1325 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:59.169147 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:59.310129 1325 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:59.310236 1325 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.834s (starting election) I0507 20:09:59.310338 1325 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:59.310777 1325 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:59.311018 1325 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:59.312034 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:09:59.482205 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.748s (SENT) I0507 20:09:59.482305 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:09:59.482596 1325 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.412s (election complete) I0507 20:09:59.482697 1325 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:09:59.556452 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.387s (SENT) I0507 20:09:59.556550 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:59.556924 1325 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.647s (election complete) I0507 20:09:59.557080 1325 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:09:59.700968 1325 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:59.701155 1325 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.473s (starting election) I0507 20:09:59.701344 1325 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:09:59.701771 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.682s (SENT) I0507 20:09:59.701869 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:09:59.702069 1325 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:59.702389 1325 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:09:59.702666 1325 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.403s (election complete) I0507 20:09:59.702795 1325 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:09:59.703505 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:09:59.896119 1326 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:09:59.896240 1326 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.630s (starting election) I0507 20:09:59.896328 1326 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:09:59.896793 1326 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:09:59.896993 1326 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:09:59.897925 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:00.106412 1326 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:00.106528 1326 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.338s (starting election) I0507 20:10:00.106616 1326 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:00.107098 1326 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:00.107266 1326 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:00.108358 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:00.146231 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.834s (SENT) I0507 20:10:00.146335 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:00.146661 1326 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.457s (election complete) I0507 20:10:00.146761 1326 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:00.176991 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.473s (SENT) I0507 20:10:00.177098 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:00.177479 1326 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.199s (election complete) I0507 20:10:00.177582 1326 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:00.204664 1326 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:00.204808 1326 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.617s (starting election) I0507 20:10:00.204949 1326 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:00.205548 1326 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:00.205778 1326 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:00.206665 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:00.377357 1326 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:00.377483 1326 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.531s (starting election) I0507 20:10:00.377575 1326 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:00.378079 1326 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:00.378273 1326 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:00.379081 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:00.446151 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.338s (SENT) I0507 20:10:00.446256 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:00.446559 1326 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.454s (election complete) I0507 20:10:00.446651 1326 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:00.529211 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.630s (SENT) I0507 20:10:00.529335 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:00.529675 1326 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.874s (election complete) I0507 20:10:00.529814 1326 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:00.822185 1326 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:00.822324 1326 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.190s (starting election) I0507 20:10:00.822434 1326 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:00.823016 1326 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:00.823211 1326 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:00.824024 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:00.824661 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.617s (SENT) I0507 20:10:00.824723 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:00.824965 1326 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.894s (election complete) I0507 20:10:00.840762 1326 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:00.901283 1326 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:00.901443 1326 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.923s (starting election) I0507 20:10:00.901554 1326 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:00.902087 1326 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:00.902273 1326 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:00.903050 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:00.910126 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.531s (SENT) I0507 20:10:00.910228 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:00.910564 1326 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.424s (election complete) I0507 20:10:00.910650 1326 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:01.013415 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.190s (SENT) I0507 20:10:01.013499 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:01.013743 1326 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.100s (election complete) I0507 20:10:01.013811 1326 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:01.114425 1326 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:01.114534 1326 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.087s (starting election) I0507 20:10:01.114611 1326 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:01.115074 1326 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:01.115233 1326 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:01.116174 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:01.201957 1326 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:01.202075 1326 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.535s (starting election) I0507 20:10:01.202152 1326 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:01.202577 1326 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:01.202759 1326 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:01.202991 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.087s (SENT) I0507 20:10:01.203052 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:01.203313 1326 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.537s (election complete) I0507 20:10:01.203358 1326 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:01.203711 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:01.335707 1327 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:01.335861 1327 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.523s (starting election) I0507 20:10:01.335955 1327 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:01.336504 1327 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:01.336697 1327 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:01.337656 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:01.404467 1327 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:01.404594 1327 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.881s (starting election) I0507 20:10:01.404687 1327 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:01.405170 1327 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:01.405345 1327 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:01.406242 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:01.571527 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:09:58.573412 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:09:58.573617 (+ 205us) negotiation.cc:98] Waiting for socket to connect 0507 20:09:58.573628 (+ 11us) client_negotiation.cc:160] Beginning negotiation 0507 20:09:58.573735 (+ 107us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:01.571169 (+2997434us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":59} W0507 20:10:01.571862 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:10:01.738973 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.535s (SENT) I0507 20:10:01.739066 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:01.739415 1327 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.403s (election complete) I0507 20:10:01.739560 1327 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:01.827144 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.923s (SENT) I0507 20:10:01.827251 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:01.827608 1327 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.093s (election complete) I0507 20:10:01.827714 1327 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:01.862187 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.523s (SENT) I0507 20:10:01.862295 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:01.862756 1327 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.523s (election complete) I0507 20:10:01.862874 1327 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:02.143657 1327 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:02.143772 1327 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.177s (starting election) I0507 20:10:02.143877 1327 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:02.144290 1327 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:02.144429 1327 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:02.145738 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:02.286870 1327 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:02.286983 1327 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.369s (starting election) I0507 20:10:02.287055 1327 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:02.287451 1327 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:02.287606 1327 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:02.288045 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.881s (SENT) I0507 20:10:02.288156 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:10:02.288197 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:02.288319 1327 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.915s (election complete) I0507 20:10:02.288352 1327 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:02.321792 1327 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:10:02.321869 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.177s (SENT) I0507 20:10:02.321897 1327 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.133s (starting election) I0507 20:10:02.321949 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:02.321967 1327 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:02.322396 1327 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:02.322551 1327 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:02.322589 1331 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.127s (election complete) I0507 20:10:02.322700 1331 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:02.323498 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:02.386515 1331 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:02.386652 1331 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.836s (starting election) I0507 20:10:02.386747 1331 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:02.387291 1331 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:02.387478 1331 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:02.388298 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:02.450907 1331 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:02.451045 1331 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.942s (starting election) I0507 20:10:02.451133 1331 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:02.451608 1331 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:02.451772 1331 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:02.452962 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:02.456295 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.133s (SENT) I0507 20:10:02.456387 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:02.456955 1331 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.286s (election complete) I0507 20:10:02.457078 1331 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:02.658587 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.369s (SENT) I0507 20:10:02.658684 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:02.659088 1331 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.204s (election complete) I0507 20:10:02.659219 1331 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:02.743664 1331 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:02.743801 1331 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.638s (starting election) I0507 20:10:02.743894 1331 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:02.744406 1331 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:02.744593 1331 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:02.745501 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:02.863965 1331 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:02.864125 1331 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.217s (starting election) I0507 20:10:02.864218 1331 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:02.864697 1331 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:02.864878 1331 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:02.865669 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:03.082248 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.217s (SENT) I0507 20:10:03.082366 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:03.082510 1331 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:03.082609 1331 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.174s (starting election) I0507 20:10:03.082712 1331 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:03.083226 1331 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:03.083417 1331 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:03.083521 1331 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.140s (election complete) I0507 20:10:03.083552 1331 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:03.084184 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:03.223860 1332 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:03.223992 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.637s (starting election) I0507 20:10:03.224098 1332 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:03.224568 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.836s (SENT) I0507 20:10:03.224660 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:03.224669 1331 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:03.224800 1331 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.163s (starting election) I0507 20:10:03.224872 1331 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:03.225239 1331 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:03.225303 1333 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.309s (election complete) I0507 20:10:03.225353 1333 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:03.225400 1331 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:03.224709 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:03.226143 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:03.226305 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:03.226395 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:03.258942 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.174s (SENT) I0507 20:10:03.259057 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:03.259363 1332 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.894s (election complete) I0507 20:10:03.259441 1332 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:03.382642 1332 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:03.382766 1332 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.177s (starting election) I0507 20:10:03.382848 1332 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:03.383307 1332 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:03.383474 1332 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:03.383702 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.638s (SENT) I0507 20:10:03.383762 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:03.383952 1332 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.466s (election complete) I0507 20:10:03.383996 1332 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:03.384238 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:03.389405 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.163s (SENT) I0507 20:10:03.389470 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:03.389672 1332 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.302s (election complete) I0507 20:10:03.389746 1332 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:03.395521 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.942s (SENT) I0507 20:10:03.395594 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:03.395812 1332 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.704s (election complete) I0507 20:10:03.395872 1332 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:03.535257 1332 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:03.535374 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.981s (starting election) I0507 20:10:03.535451 1332 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:03.535893 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:03.536048 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:03.536944 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:03.561010 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.177s (SENT) I0507 20:10:03.561101 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:03.561359 1332 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.624s (election complete) I0507 20:10:03.561429 1332 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:03.693017 1332 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:03.693156 1332 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.859s (starting election) I0507 20:10:03.693254 1332 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:03.693827 1332 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:03.694022 1332 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:03.694857 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:03.863999 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.637s (SENT) I0507 20:10:03.864102 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:03.864403 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.113s (election complete) I0507 20:10:03.864490 1332 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:03.978729 1332 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:03.978838 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.958s (starting election) I0507 20:10:03.978935 1332 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:03.979368 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:03.979524 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:03.980465 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:04.186619 1332 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:04.186744 1332 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.822s (starting election) I0507 20:10:04.186834 1332 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:04.187284 1332 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:04.187446 1332 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:04.188258 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:04.519237 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.981s (SENT) I0507 20:10:04.519366 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:04.519752 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.433s (election complete) I0507 20:10:04.519847 1332 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:04.552685 1332 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:04.552793 1332 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.468s (starting election) I0507 20:10:04.552861 1332 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:04.553300 1332 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:04.553445 1332 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:04.553659 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.859s (SENT) I0507 20:10:04.553707 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:04.554005 1332 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.534s (election complete) I0507 20:10:04.554098 1332 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:04.554159 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:04.938323 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.958s (SENT) I0507 20:10:04.938428 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:04.938813 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.135s (election complete) I0507 20:10:04.938943 1332 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:05.009944 1332 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:05.010090 1332 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.563s (starting election) I0507 20:10:05.010185 1332 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:05.010680 1332 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:05.010910 1332 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:05.011180 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.822s (SENT) I0507 20:10:05.011266 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:05.011489 1332 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.905s (election complete) I0507 20:10:05.011538 1332 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:05.011724 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:05.022065 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.468s (SENT) I0507 20:10:05.022158 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:05.022456 1332 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.855s (election complete) I0507 20:10:05.022555 1332 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:05.074810 1332 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:05.074973 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.218s (starting election) I0507 20:10:05.075073 1332 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:05.075642 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:05.075851 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:05.076961 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:05.291517 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:02.293308 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:02.293617 (+ 309us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:02.293630 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:02.293728 (+ 98us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:05.291235 (+2997507us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":63} W0507 20:10:05.291864 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:10:05.294322 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.218s (SENT) I0507 20:10:05.294442 1332 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:05.294548 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:05.294559 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.950s (starting election) I0507 20:10:05.294648 1332 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:05.295174 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:05.295346 1332 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:05.295433 1332 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.778s (election complete) I0507 20:10:05.295460 1332 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:05.296183 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:05.575282 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.563s (SENT) I0507 20:10:05.575377 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:05.575773 1334 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.176s (election complete) I0507 20:10:05.575875 1334 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:05.752905 1334 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:05.753041 1334 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.722s (starting election) I0507 20:10:05.753171 1334 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:05.753670 1334 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:05.753867 1334 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:05.754860 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:05.879115 1334 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:05.879262 1334 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.241s (starting election) I0507 20:10:05.879350 1334 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:05.879900 1334 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:05.880084 1334 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:05.881114 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:06.074348 1334 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:06.074487 1334 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.642s (starting election) I0507 20:10:06.074586 1334 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:06.075122 1334 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:06.075304 1334 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:06.076292 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:06.121232 1334 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:06.121412 1334 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.620s (starting election) I0507 20:10:06.121532 1334 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:06.122195 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.241s (SENT) I0507 20:10:06.122272 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:06.122294 1334 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:06.122539 1334 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:06.122710 1334 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.936s (election complete) I0507 20:10:06.122766 1334 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:06.123389 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:06.247041 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.950s (SENT) I0507 20:10:06.247128 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:06.247462 1337 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.905s (election complete) I0507 20:10:06.247731 1337 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:06.476153 1337 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:06.476289 1337 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.224s (starting election) I0507 20:10:06.476380 1337 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:06.476956 1337 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:06.477152 1337 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:06.477262 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.722s (SENT) I0507 20:10:06.477396 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:06.477614 1337 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.363s (election complete) I0507 20:10:06.477740 1337 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:06.477985 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:06.702648 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.224s (SENT) I0507 20:10:06.702749 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:06.703147 1337 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.978s (election complete) I0507 20:10:06.703253 1337 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:06.718547 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.642s (SENT) I0507 20:10:06.718641 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:06.718991 1337 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.557s (election complete) I0507 20:10:06.719110 1337 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:06.744598 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.620s (SENT) I0507 20:10:06.744691 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:06.744999 1337 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.225s (election complete) I0507 20:10:06.745103 1337 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:06.970902 1337 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:06.971040 1337 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.716s (starting election) I0507 20:10:06.971140 1337 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:06.971678 1337 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:06.971873 1337 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:06.972630 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:07.276914 1337 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:07.277055 1337 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.281s (starting election) I0507 20:10:07.277155 1337 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:07.277647 1337 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:07.277873 1337 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:07.278787 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:07.558756 1337 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:07.558915 1337 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.539s (starting election) I0507 20:10:07.559007 1337 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:07.559506 1337 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:07.559602 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.281s (SENT) I0507 20:10:07.559702 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:07.559728 1337 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:07.559937 1337 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.274s (election complete) I0507 20:10:07.559986 1337 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:07.560487 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:07.682229 1337 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:07.682404 1337 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.711s (starting election) I0507 20:10:07.682544 1337 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:07.683262 1337 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:07.683518 1337 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:07.684592 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:07.688252 1337 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:07.688357 1337 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.924s (starting election) I0507 20:10:07.688460 1337 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:07.689039 1337 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:07.689214 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.716s (SENT) I0507 20:10:07.689240 1337 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:07.689272 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:07.689496 1337 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.683s (election complete) I0507 20:10:07.689604 1337 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:07.689899 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:07.835063 1337 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:07.835278 1337 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.516s (starting election) I0507 20:10:07.835407 1337 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:07.836063 1337 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:07.836330 1337 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:07.837144 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:08.101178 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.539s (SENT) I0507 20:10:08.101276 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:08.101719 1337 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.867s (election complete) I0507 20:10:08.101855 1337 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:08.354455 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.516s (SENT) I0507 20:10:08.354539 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:08.354956 1337 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.609s (election complete) I0507 20:10:08.355108 1337 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:08.373862 1337 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:08.374061 1337 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.180s (starting election) I0507 20:10:08.374207 1337 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:08.374832 1337 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:08.375100 1337 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:08.376118 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:08.395007 1337 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:08.395171 1337 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.144s (starting election) I0507 20:10:08.395278 1337 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:08.395850 1337 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:08.396073 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.711s (SENT) I0507 20:10:08.396075 1337 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:08.396200 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:08.396463 1337 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.616s (election complete) I0507 20:10:08.396522 1337 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:08.397192 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:08.541256 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.144s (SENT) I0507 20:10:08.541368 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:08.541748 1337 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.734s (election complete) I0507 20:10:08.541887 1337 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:08.555531 1337 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:08.555683 1337 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.271s (starting election) I0507 20:10:08.555819 1337 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:08.556171 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.180s (SENT) I0507 20:10:08.556251 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:08.556602 1337 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:08.556824 1337 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:08.556957 1337 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.862s (election complete) I0507 20:10:08.556998 1337 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:08.557703 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:08.614590 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.924s (SENT) I0507 20:10:08.614678 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:08.615053 1339 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.098s (election complete) I0507 20:10:08.615285 1339 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:08.713645 1339 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:08.713786 1339 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.722s (starting election) I0507 20:10:08.713883 1339 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:08.714409 1339 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:08.714623 1339 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:08.715476 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:08.828577 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.271s (SENT) I0507 20:10:08.828675 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:08.829017 1339 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.760s (election complete) I0507 20:10:08.829120 1339 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:08.965041 1339 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:08.965173 1339 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.661s (starting election) I0507 20:10:08.965255 1339 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:08.965683 1339 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:08.965840 1339 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:08.966547 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:09.019577 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:06.020011 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:06.020228 (+ 217us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:06.020241 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:06.020334 (+ 93us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:09.019229 (+2998895us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":65} W0507 20:10:09.019841 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:10:09.277034 1339 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:09.277142 1339 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.156s (starting election) I0507 20:10:09.277221 1339 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:09.277660 1339 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:09.277827 1339 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:09.278707 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:09.433650 1339 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:09.433763 1339 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.338s (starting election) I0507 20:10:09.433840 1339 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:09.434317 1339 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:09.434495 1339 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:09.434787 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.156s (SENT) I0507 20:10:09.434870 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:09.435173 1339 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.413s (election complete) I0507 20:10:09.435266 1339 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:09.435322 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:09.437721 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.722s (SENT) I0507 20:10:09.437793 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:09.437995 1339 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.929s (election complete) I0507 20:10:09.438046 1339 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:09.626713 1339 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:09.626819 1339 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.842s (starting election) I0507 20:10:09.626909 1339 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:09.627331 1339 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:09.627466 1339 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:09.628253 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:09.628979 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.661s (SENT) I0507 20:10:09.629034 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:09.629204 1339 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.235s (election complete) I0507 20:10:09.629264 1339 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:09.773190 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.338s (SENT) I0507 20:10:09.773281 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:09.773579 1339 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.830s (election complete) I0507 20:10:09.773677 1339 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:09.865461 1339 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:09.865591 1339 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.815s (starting election) I0507 20:10:09.865680 1339 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:09.866124 1339 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:09.866294 1339 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:09.867012 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:10.368046 1342 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:10.368144 1342 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.378s (starting election) I0507 20:10:10.368270 1342 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:10.368671 1342 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:10.368839 1342 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:10.369637 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:10.471290 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.842s (SENT) I0507 20:10:10.471407 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:10.471782 1342 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.502s (election complete) I0507 20:10:10.471894 1342 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:10.604961 1342 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:10.605085 1342 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.284s (starting election) I0507 20:10:10.605175 1342 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:10.605605 1342 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:10.605772 1342 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:10.606624 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:10.682425 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.815s (SENT) I0507 20:10:10.682518 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:10.682838 1342 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.893s (election complete) I0507 20:10:10.682929 1342 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:10.746630 1342 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:10.746767 1342 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.342s (starting election) I0507 20:10:10.746899 1342 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:10.747449 1342 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:10.747704 1342 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:10.747715 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.378s (SENT) I0507 20:10:10.747812 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:10.748109 1342 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.532s (election complete) I0507 20:10:10.748236 1342 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:10.748488 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:10.890568 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.284s (SENT) I0507 20:10:10.890558 1342 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:10.890688 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:10.890724 1342 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.357s (starting election) I0507 20:10:10.890836 1342 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:10.891438 1342 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:10.891623 1342 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:10.891774 1342 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.234s (election complete) I0507 20:10:10.891815 1342 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:10.892413 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:11.090895 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.342s (SENT) I0507 20:10:11.091003 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:11.091325 1344 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.540s (election complete) I0507 20:10:11.091428 1344 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:11.126493 1344 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:11.126662 1344 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.724s (starting election) I0507 20:10:11.126767 1344 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:11.127405 1344 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:11.127651 1344 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:11.128547 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:11.249990 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.357s (SENT) I0507 20:10:11.250105 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:11.250496 1344 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.595s (election complete) I0507 20:10:11.250603 1344 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:11.576877 1344 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:11.577044 1344 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.148s (starting election) I0507 20:10:11.577164 1344 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:11.577780 1344 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:11.578016 1344 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:11.579013 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:11.632565 1344 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:11.632676 1344 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.805s (starting election) I0507 20:10:11.632757 1344 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:11.633205 1344 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:11.633357 1344 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:11.634192 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:11.725873 1344 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:11.725996 1344 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.864s (starting election) I0507 20:10:11.726073 1344 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:11.726382 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.148s (SENT) I0507 20:10:11.726444 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:11.726495 1344 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:11.726611 1344 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:11.726730 1344 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.313s (election complete) I0507 20:10:11.726792 1344 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:11.727520 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:11.846570 1345 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:11.846693 1345 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.983s (starting election) I0507 20:10:11.846792 1345 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:11.847249 1345 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:11.847420 1345 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:11.848248 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:11.853014 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.724s (SENT) I0507 20:10:11.853094 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:11.853395 1345 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.635s (election complete) I0507 20:10:11.853468 1345 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:12.040385 1345 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:12.040499 1345 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.537s (starting election) I0507 20:10:12.040588 1345 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:12.041018 1345 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:12.041199 1345 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:12.041947 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:12.439016 1345 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:12.439126 1345 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.291s (starting election) I0507 20:10:12.439214 1345 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:12.439643 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.805s (SENT) I0507 20:10:12.439702 1345 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:12.439831 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:12.439942 1345 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:12.440130 1345 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.371s (election complete) I0507 20:10:12.440243 1345 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:12.440917 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:12.489152 1346 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:12.489277 1346 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.614s (starting election) I0507 20:10:12.489418 1346 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:12.489919 1346 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:12.490130 1346 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:12.490996 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:12.578266 1346 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:12.578408 1346 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.564s (starting election) I0507 20:10:12.578500 1346 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:12.578977 1346 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:12.579208 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.537s (SENT) I0507 20:10:12.579391 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:12.579783 1345 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.867s (election complete) I0507 20:10:12.579932 1345 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:12.580022 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:12.580145 1346 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:12.592774 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.864s (SENT) I0507 20:10:12.592893 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:12.593252 1346 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.746s (election complete) I0507 20:10:12.593364 1346 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:12.732311 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.291s (SENT) I0507 20:10:12.732434 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:12.732769 1346 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.702s (election complete) I0507 20:10:12.732861 1346 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:12.759701 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:09.761209 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:09.761553 (+ 344us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:09.761567 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:09.761669 (+ 102us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:12.759265 (+2997596us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":90} W0507 20:10:12.759971 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:10:12.832419 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.983s (SENT) I0507 20:10:12.832520 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:12.832873 1346 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.945s (election complete) I0507 20:10:12.832969 1346 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:13.105432 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.614s (SENT) I0507 20:10:13.105567 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:13.105984 1346 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.781s (election complete) I0507 20:10:13.106097 1346 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:13.146345 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.564s (SENT) I0507 20:10:13.146457 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:13.146796 1346 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.662s (election complete) I0507 20:10:13.146919 1346 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:13.435637 1346 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:13.435823 1346 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.780s (starting election) I0507 20:10:13.435964 1346 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:13.436624 1346 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:13.436887 1346 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:13.438129 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:13.810349 1346 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:13.810525 1346 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.657s (starting election) I0507 20:10:13.810649 1346 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:13.811388 1346 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:13.811651 1346 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:13.812764 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:13.888345 1346 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:13.888507 1346 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.603s (starting election) I0507 20:10:13.888629 1346 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:13.889389 1346 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:13.889645 1346 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:13.890770 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:14.216697 1346 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:14.216876 1346 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.140s (starting election) I0507 20:10:14.217002 1346 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:14.217470 1346 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:14.217685 1346 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:14.218112 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.780s (SENT) I0507 20:10:14.218175 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:14.218355 1346 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.914s (election complete) I0507 20:10:14.218442 1346 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:14.218809 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:14.358674 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.140s (SENT) I0507 20:10:14.358793 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:14.359148 1346 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.701s (election complete) I0507 20:10:14.359222 1346 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:14.468469 1346 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:14.468611 1346 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.827s (starting election) I0507 20:10:14.468713 1346 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:14.469334 1346 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:14.469580 1346 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:14.469930 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.657s (SENT) I0507 20:10:14.469992 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:14.470204 1346 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.649s (election complete) I0507 20:10:14.470257 1346 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:14.470393 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:14.492811 1346 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:14.492920 1346 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.777s (starting election) I0507 20:10:14.493003 1346 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:14.493417 1346 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:14.493569 1346 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:14.494320 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:14.495003 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.603s (SENT) I0507 20:10:14.495059 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:14.495211 1346 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.863s (election complete) I0507 20:10:14.495249 1346 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:15.061249 1350 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:15.061373 1350 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.705s (starting election) I0507 20:10:15.061478 1350 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:15.061967 1350 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:15.062182 1350 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:15.063293 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:15.120076 1350 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:15.120198 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.726s (starting election) I0507 20:10:15.120287 1350 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:15.120772 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:15.120944 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:15.121819 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:15.272528 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.777s (SENT) I0507 20:10:15.272642 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:15.273038 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.865s (election complete) I0507 20:10:15.273113 1350 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:15.298207 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.827s (SENT) I0507 20:10:15.298319 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:15.298672 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.674s (election complete) I0507 20:10:15.298769 1350 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:15.767181 1350 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:15.767299 1350 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.555s (starting election) I0507 20:10:15.767364 1350 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:15.767783 1350 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:15.767935 1350 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:15.768198 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.705s (SENT) I0507 20:10:15.768281 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:15.768441 1350 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.729s (election complete) I0507 20:10:15.768472 1350 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:15.768998 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:15.847770 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.726s (SENT) I0507 20:10:15.847875 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:15.848217 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.104s (election complete) I0507 20:10:15.848377 1350 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:15.952798 1350 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:15.952942 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.574s (starting election) I0507 20:10:15.953083 1350 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:15.953616 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:15.953836 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:15.954696 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:16.138809 1350 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:16.138957 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.093s (starting election) I0507 20:10:16.139063 1350 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:16.139513 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:16.139695 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:16.140771 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:16.233326 1350 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:10:16.233393 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.093s (SENT) I0507 20:10:16.233458 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:16.233443 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.670s (starting election) I0507 20:10:16.233522 1350 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:16.233963 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:16.233952 1353 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.096s (election complete) I0507 20:10:16.234007 1353 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:16.234108 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:16.235002 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:16.324352 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.555s (SENT) I0507 20:10:16.324445 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:16.324744 1350 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.295s (election complete) I0507 20:10:16.324826 1350 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:16.330771 1350 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:16.330899 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.602s (starting election) I0507 20:10:16.331012 1350 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:16.331544 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:16.331755 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:16.332770 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:16.507535 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:13.507626 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:13.507895 (+ 269us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:13.507909 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:13.508034 (+ 125us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:16.507191 (+2999157us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":86} W0507 20:10:16.507831 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:10:16.527879 1350 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:16.528007 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.486s (starting election) I0507 20:10:16.528100 1350 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:16.528645 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:16.528900 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:16.529089 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.574s (SENT) I0507 20:10:16.529161 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:16.529369 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.096s (election complete) I0507 20:10:16.529414 1350 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:16.529779 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:16.621163 1350 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:16.621294 1350 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.715s (starting election) I0507 20:10:16.621383 1350 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:16.621917 1350 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:16.622112 1350 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:16.623145 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:16.625686 1350 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:16.625761 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.969s (starting election) I0507 20:10:16.625830 1350 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:16.626271 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:16.626404 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:16.627285 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:16.905598 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.670s (SENT) I0507 20:10:16.905694 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:16.906013 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.516s (election complete) I0507 20:10:16.906136 1350 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:16.935621 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.602s (SENT) I0507 20:10:16.935712 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:16.936019 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.531s (election complete) I0507 20:10:16.936098 1350 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:17.016299 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.486s (SENT) I0507 20:10:17.016420 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:17.016763 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.096s (election complete) I0507 20:10:17.016855 1350 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:17.113353 1350 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:17.113476 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.656s (starting election) I0507 20:10:17.113590 1350 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:17.114084 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:17.114260 1350 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:17.115139 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:17.337059 1350 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:17.337173 1350 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.355s (starting election) I0507 20:10:17.337256 1350 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:17.337738 1350 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:17.337877 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.715s (SENT) I0507 20:10:17.337930 1350 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:17.337982 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:17.338238 1350 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.755s (election complete) I0507 20:10:17.338279 1350 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:17.338784 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:17.467816 1350 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:17.467921 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.477s (starting election) I0507 20:10:17.468026 1350 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:17.468430 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:17.468569 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:17.469316 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:17.597167 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.969s (SENT) I0507 20:10:17.597267 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:17.597534 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.418s (election complete) I0507 20:10:17.597604 1350 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:17.693590 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.355s (SENT) I0507 20:10:17.693706 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:17.694120 1350 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.419s (election complete) I0507 20:10:17.694214 1350 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:17.771739 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.656s (SENT) I0507 20:10:17.771842 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:17.772223 1350 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.499s (election complete) I0507 20:10:17.772313 1350 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:17.946179 1350 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:10:17.946256 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.477s (SENT) I0507 20:10:17.946286 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.276s (starting election) I0507 20:10:17.946318 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:17.946352 1350 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:17.946759 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:17.946914 1350 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:17.947015 1350 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.132s (election complete) I0507 20:10:17.947051 1350 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:17.947665 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:18.080022 1356 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:18.080178 1356 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.704s (starting election) I0507 20:10:18.080257 1356 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:18.080705 1356 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:18.080847 1356 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:18.081578 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:18.113608 1356 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:18.113731 1356 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.374s (starting election) I0507 20:10:18.113827 1356 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:18.114302 1356 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:18.114516 1356 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:18.115278 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:18.224047 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.276s (SENT) I0507 20:10:18.224161 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:18.224510 1356 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.490s (election complete) I0507 20:10:18.224603 1356 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:18.271831 1356 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:18.271930 1356 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.317s (starting election) I0507 20:10:18.272003 1356 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:18.272398 1356 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:18.272531 1356 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:18.273128 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:18.488587 1356 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:18.488713 1356 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.320s (starting election) I0507 20:10:18.488791 1356 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:18.489094 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.374s (SENT) I0507 20:10:18.489168 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:18.489233 1356 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:18.489420 1356 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:18.489509 1356 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.927s (election complete) I0507 20:10:18.489547 1356 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:18.490442 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:18.589987 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.317s (SENT) I0507 20:10:18.590070 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:18.590065 1357 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:18.590190 1357 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.554s (starting election) I0507 20:10:18.590406 1357 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:18.590831 1357 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:18.590850 1356 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.982s (election complete) I0507 20:10:18.590911 1356 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:18.590993 1357 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:18.591637 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:18.715488 1357 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:18.715591 1357 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.870s (starting election) I0507 20:10:18.715659 1357 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:18.716040 1357 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:18.716213 1357 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:18.716876 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:18.785890 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.704s (SENT) I0507 20:10:18.785987 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:18.786286 1357 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.184s (election complete) I0507 20:10:18.786361 1357 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:18.810173 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.320s (SENT) I0507 20:10:18.810267 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:18.810575 1357 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.388s (election complete) I0507 20:10:18.810647 1357 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:18.971449 1357 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:18.971555 1357 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.395s (starting election) I0507 20:10:18.971634 1357 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:18.972065 1357 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:18.972206 1357 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:18.972996 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:19.145987 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.554s (SENT) I0507 20:10:19.146108 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:19.146435 1357 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.198s (election complete) I0507 20:10:19.146509 1357 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:19.199558 1357 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:19.199668 1357 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.281s (starting election) I0507 20:10:19.199748 1357 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:19.200219 1357 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:19.200387 1357 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:19.201267 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:19.344949 1357 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:19.345067 1357 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.780s (starting election) I0507 20:10:19.345144 1357 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:19.345613 1357 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:19.345753 1357 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:19.346735 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:19.367714 1357 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:10:19.367751 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.395s (SENT) I0507 20:10:19.367854 1357 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.987s (starting election) I0507 20:10:19.367899 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:19.367947 1357 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:19.368367 1357 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:19.368546 1357 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:19.368527 1358 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.430s (election complete) I0507 20:10:19.369267 1358 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:19.369467 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:19.481850 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.281s (SENT) I0507 20:10:19.481937 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:19.481925 1358 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:19.482046 1358 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.263s (starting election) I0507 20:10:19.482144 1358 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:19.482630 1358 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:19.482669 1357 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.881s (election complete) I0507 20:10:19.482733 1357 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:19.482851 1358 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:19.483443 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:19.587883 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.870s (SENT) I0507 20:10:19.587997 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:19.588433 1358 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.402s (election complete) I0507 20:10:19.588546 1358 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:19.747423 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.263s (SENT) I0507 20:10:19.747563 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:19.747982 1358 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.491s (election complete) I0507 20:10:19.748091 1358 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:19.991122 1358 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:19.991271 1358 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.894s (starting election) I0507 20:10:19.991384 1358 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:19.991936 1358 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:19.992169 1358 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:19.993440 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:20.126257 1358 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:20.126392 1358 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.152s (starting election) I0507 20:10:20.126477 1358 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:20.126937 1358 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:20.127102 1358 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:20.127319 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.780s (SENT) I0507 20:10:20.127364 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:10:20.127837 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:20.127893 1360 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.179s (election complete) I0507 20:10:20.127988 1360 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:20.240422 1360 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:20.240573 1360 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.418s (starting election) I0507 20:10:20.240691 1360 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:20.241253 1360 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:20.241446 1360 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:20.242246 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:20.263567 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:17.263417 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:17.263679 (+ 262us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:17.263692 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:17.263786 (+ 94us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:20.263254 (+2999468us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":97} W0507 20:10:20.263896 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:10:20.280323 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.152s (SENT) I0507 20:10:20.280437 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:20.280781 1360 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.493s (election complete) I0507 20:10:20.280879 1360 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:20.357105 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.987s (SENT) I0507 20:10:20.357255 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:20.357597 1360 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.526s (election complete) I0507 20:10:20.357697 1360 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:20.658999 1360 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:20.659153 1360 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.859s (starting election) I0507 20:10:20.659252 1360 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:20.659755 1360 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:20.659945 1360 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:20.660404 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.418s (SENT) I0507 20:10:20.660567 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:10:20.660771 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:20.660892 1360 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.543s (election complete) I0507 20:10:20.660965 1360 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:20.774358 1360 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:20.774482 1360 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.342s (starting election) I0507 20:10:20.774572 1360 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:20.775102 1360 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:20.775333 1360 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:20.776173 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:20.885071 1360 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:20.885205 1360 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.114s (starting election) I0507 20:10:20.885299 1360 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:20.885799 1360 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:20.886004 1360 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:20.886920 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:20.887702 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.894s (SENT) I0507 20:10:20.887814 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:20.888105 1360 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.694s (election complete) I0507 20:10:20.888221 1360 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:21.000993 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.114s (SENT) I0507 20:10:21.001116 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:21.001536 1360 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.521s (election complete) I0507 20:10:21.001667 1360 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:21.117985 1360 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:21.118142 1360 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.525s (starting election) I0507 20:10:21.118304 1360 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:21.118516 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.342s (SENT) I0507 20:10:21.118636 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:21.118836 1360 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:21.119038 1360 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:21.119194 1360 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.918s (election complete) I0507 20:10:21.119258 1360 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:21.120100 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:21.204819 1360 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:21.204952 1360 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.762s (starting election) I0507 20:10:21.205060 1360 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:21.205576 1360 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:21.205822 1360 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:21.206835 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:21.520547 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.859s (SENT) I0507 20:10:21.520654 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:21.521100 1360 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.635s (election complete) I0507 20:10:21.521268 1360 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:21.523429 1360 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:21.523551 1360 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.199s (starting election) I0507 20:10:21.523655 1360 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:21.524211 1360 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:21.524440 1360 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:21.525511 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:21.645905 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.525s (SENT) I0507 20:10:21.646016 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:21.646528 1360 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.485s (election complete) I0507 20:10:21.646677 1360 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:21.723758 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.199s (SENT) I0507 20:10:21.723762 1360 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:21.723877 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:21.723917 1360 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.087s (starting election) I0507 20:10:21.724036 1360 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:21.724576 1360 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:21.724819 1360 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:21.724963 1360 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.534s (election complete) I0507 20:10:21.725023 1360 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:21.725791 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:21.812608 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.087s (SENT) I0507 20:10:21.812714 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:21.813136 1364 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.656s (election complete) I0507 20:10:21.813266 1364 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:21.969655 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.762s (SENT) I0507 20:10:21.969764 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:21.970229 1364 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.964s (election complete) I0507 20:10:21.970352 1364 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:22.132699 1364 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:22.132854 1364 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.669s (starting election) I0507 20:10:22.132968 1364 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:22.133512 1364 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:22.133756 1364 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:22.134716 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:22.470438 1364 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:22.470594 1364 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.083s (starting election) I0507 20:10:22.470736 1364 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:22.471277 1364 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:22.471490 1364 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:22.472375 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:22.554486 1364 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:22.554649 1364 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.451s (starting election) I0507 20:10:22.554785 1364 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:22.555491 1364 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:22.555619 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.083s (SENT) I0507 20:10:22.555729 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:22.555734 1364 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:22.556002 1364 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.132s (election complete) I0507 20:10:22.556071 1364 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:22.556730 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:22.689420 1364 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:22.689576 1364 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.462s (starting election) I0507 20:10:22.689705 1364 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:22.690227 1364 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:22.690435 1364 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:22.691449 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:22.803146 1364 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:22.803366 1364 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.380s (starting election) W0507 20:10:22.803483 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.669s (SENT) I0507 20:10:22.803602 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:22.803530 1364 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:22.804285 1364 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:22.804572 1364 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:22.804788 1364 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.954s (election complete) I0507 20:10:22.804877 1364 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:22.805446 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:22.935703 1365 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:22.935863 1365 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.985s (starting election) I0507 20:10:22.935988 1365 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:22.936563 1365 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:22.936769 1365 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:22.937655 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:23.008018 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.451s (SENT) I0507 20:10:23.008110 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:23.008419 1365 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.243s (election complete) I0507 20:10:23.008523 1365 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:23.153410 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.462s (SENT) I0507 20:10:23.153522 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:23.153880 1365 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.928s (election complete) I0507 20:10:23.153980 1365 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:23.186154 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.380s (SENT) I0507 20:10:23.186254 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:23.186621 1365 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.101s (election complete) I0507 20:10:23.186707 1365 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:23.288763 1365 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:23.288877 1365 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.364s (starting election) I0507 20:10:23.288970 1365 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:23.289434 1365 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:23.289613 1365 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:23.290529 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:23.653789 1365 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:23.653900 1365 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.627s (starting election) I0507 20:10:23.653988 1365 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:23.654422 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.364s (SENT) I0507 20:10:23.654454 1365 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:23.654496 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:23.654639 1365 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:23.654762 1365 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.114s (election complete) I0507 20:10:23.654811 1365 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:23.655906 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:23.769938 1366 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:23.770068 1366 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.849s (starting election) I0507 20:10:23.770174 1366 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:23.770742 1366 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:23.771580 1366 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:23.771958 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:23.921969 1366 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:23.922099 1366 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.665s (starting election) I0507 20:10:23.922240 1366 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:23.922683 1366 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:23.922864 1366 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:23.923303 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.985s (SENT) I0507 20:10:23.923401 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:23.923713 1366 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.462s (election complete) W0507 20:10:23.923758 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:23.923808 1366 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:24.031500 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:21.033233 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:21.033548 (+ 315us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:21.033567 (+ 19us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:21.033677 (+ 110us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:24.031148 (+2997471us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":85} W0507 20:10:24.031777 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:10:24.083082 1366 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:24.083194 1366 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.940s (starting election) I0507 20:10:24.083298 1366 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:24.083742 1366 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:24.083919 1366 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:24.084703 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:24.283216 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.627s (SENT) I0507 20:10:24.283330 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:24.283632 1366 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.913s (election complete) I0507 20:10:24.283715 1366 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:24.387014 1366 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:24.387123 1366 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.145s (starting election) I0507 20:10:24.387217 1366 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:24.387660 1366 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:24.387802 1366 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:24.388649 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:24.533375 1366 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:24.533517 1366 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.588s (starting election) I0507 20:10:24.533646 1366 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:24.533774 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.145s (SENT) I0507 20:10:24.533890 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:24.534087 1366 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:24.534297 1366 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:24.534446 1366 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.846s (election complete) I0507 20:10:24.534495 1366 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:24.535362 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:24.589414 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.665s (SENT) I0507 20:10:24.589510 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:24.589781 1366 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.361s (election complete) I0507 20:10:24.589862 1366 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:24.622123 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.849s (SENT) I0507 20:10:24.622261 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:24.622543 1366 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.446s (election complete) I0507 20:10:24.622650 1366 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:24.951874 1366 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:24.951987 1366 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.886s (starting election) I0507 20:10:24.952067 1366 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:24.952503 1366 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:24.952672 1366 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:24.953676 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:25.024116 1366 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:25.024217 1366 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.984s (starting election) I0507 20:10:25.024284 1366 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:25.024724 1366 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:25.024871 1366 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:25.025046 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.940s (SENT) I0507 20:10:25.025105 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:25.025300 1366 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.627s (election complete) I0507 20:10:25.025352 1366 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:25.025844 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:25.069397 1366 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:25.069521 1366 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.752s (starting election) I0507 20:10:25.069604 1366 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:25.070039 1366 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:25.070250 1366 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:25.071156 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:25.123766 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.588s (SENT) I0507 20:10:25.123858 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:25.124181 1366 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.764s (election complete) I0507 20:10:25.124300 1366 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:25.653344 1370 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:25.653484 1370 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.126s (starting election) I0507 20:10:25.653584 1370 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:25.654091 1370 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:25.654295 1370 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:25.655400 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:25.780966 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.126s (SENT) I0507 20:10:25.780946 1370 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:25.781070 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:25.781095 1370 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.238s (starting election) I0507 20:10:25.781196 1370 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:25.781725 1370 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:25.781755 1372 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.870s (election complete) I0507 20:10:25.781838 1372 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:25.781898 1370 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:25.782644 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:25.822293 1370 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:25.822407 1370 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.371s (starting election) I0507 20:10:25.822487 1370 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:25.822928 1370 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:25.823086 1370 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:25.823307 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.752s (SENT) I0507 20:10:25.823385 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:25.823567 1370 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.897s (election complete) I0507 20:10:25.823618 1370 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:25.824019 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:25.840796 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.886s (SENT) I0507 20:10:25.840886 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:25.841240 1370 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.910s (election complete) I0507 20:10:25.841317 1370 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:26.009886 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.984s (SENT) I0507 20:10:26.009984 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:26.010270 1370 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.215s (election complete) I0507 20:10:26.010339 1370 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:26.021004 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.238s (SENT) I0507 20:10:26.021106 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:26.021361 1370 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.110s (election complete) I0507 20:10:26.021427 1370 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:26.132519 1370 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:26.132623 1370 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.407s (starting election) I0507 20:10:26.132702 1370 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:26.133162 1370 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:26.133327 1370 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:26.134330 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:26.195194 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.371s (SENT) I0507 20:10:26.195307 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:26.195612 1370 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.351s (election complete) I0507 20:10:26.195677 1370 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:26.540815 1370 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:10:26.540889 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.407s (SENT) I0507 20:10:26.540915 1370 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.563s (starting election) I0507 20:10:26.540951 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:26.540985 1370 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:26.541410 1370 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:26.541601 1370 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:26.541574 1373 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.475s (election complete) I0507 20:10:26.541659 1373 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:26.542608 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:26.547714 1373 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:26.547832 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.796s (starting election) I0507 20:10:26.547924 1373 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:26.548439 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:26.548658 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:26.549538 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:26.752666 1373 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:26.752807 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.779s (starting election) I0507 20:10:26.752903 1373 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:26.753500 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:26.753688 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:26.754695 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:27.018097 1373 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:27.018221 1373 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.961s (starting election) I0507 20:10:27.018378 1373 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:27.018805 1373 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:27.019003 1373 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:27.019834 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:27.105978 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.563s (SENT) I0507 20:10:27.106101 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:27.106508 1373 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.620s (election complete) I0507 20:10:27.106668 1373 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:27.344882 1373 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:27.345016 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.124s (starting election) I0507 20:10:27.345113 1373 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:27.345578 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:27.345680 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.796s (SENT) I0507 20:10:27.345777 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:27.345798 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:27.346117 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.149s (election complete) I0507 20:10:27.346194 1373 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:27.346657 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:27.470772 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.124s (SENT) I0507 20:10:27.470963 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:27.471402 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.344s (election complete) I0507 20:10:27.471509 1373 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:27.532449 1373 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:27.532574 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.176s (starting election) I0507 20:10:27.532675 1373 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:27.533227 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:27.533413 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:27.533752 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.779s (SENT) I0507 20:10:27.533848 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:27.534042 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.097s (election complete) I0507 20:10:27.534113 1373 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:27.534230 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:27.632200 1373 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:27.632323 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.960s (starting election) I0507 20:10:27.632413 1373 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:27.632870 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:27.633072 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:27.634017 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:27.710606 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.176s (SENT) I0507 20:10:27.710726 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:27.711119 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.929s (election complete) I0507 20:10:27.711206 1373 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:27.727753 1373 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:27.727869 1373 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.600s (starting election) I0507 20:10:27.727955 1373 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:27.728447 1373 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:27.728629 1373 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:27.729594 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:27.807421 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:24.809590 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:24.809865 (+ 275us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:24.809879 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:24.809977 (+ 98us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:27.807147 (+2997170us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":93} W0507 20:10:27.807688 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:10:27.816130 1373 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:27.816254 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.505s (starting election) I0507 20:10:27.816346 1373 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:27.816865 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:27.817068 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:27.817993 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:27.981010 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.961s (SENT) I0507 20:10:27.981119 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:27.981479 1373 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.244s (election complete) I0507 20:10:27.981568 1373 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:28.226132 1373 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:28.226246 1373 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.702s (starting election) I0507 20:10:28.226330 1373 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:28.226797 1373 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:28.226997 1373 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:28.227947 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:28.322695 1373 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:28.322823 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.151s (starting election) I0507 20:10:28.322933 1373 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:28.323416 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:28.323626 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:28.323868 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.505s (SENT) I0507 20:10:28.323976 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:28.324301 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.791s (election complete) I0507 20:10:28.324374 1373 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:28.324477 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:28.330111 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.600s (SENT) I0507 20:10:28.330193 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:28.330440 1373 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.780s (election complete) I0507 20:10:28.330534 1373 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:28.475858 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.151s (SENT) I0507 20:10:28.475986 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:28.476311 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.868s (election complete) I0507 20:10:28.476392 1373 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:28.594180 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.960s (SENT) I0507 20:10:28.594280 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:28.594560 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.260s (election complete) I0507 20:10:28.594671 1373 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:28.855947 1373 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:28.856052 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.704s (starting election) I0507 20:10:28.856135 1373 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:28.856570 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:28.856734 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:28.857878 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:28.930459 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.702s (SENT) I0507 20:10:28.930577 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:28.930982 1373 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.804s (election complete) I0507 20:10:28.931102 1373 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:29.345952 1373 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:29.346077 1373 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.616s (starting election) I0507 20:10:29.346171 1373 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:29.346673 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:29.346901 1373 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:29.347823 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:29.561511 1373 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:29.561638 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.570s (starting election) I0507 20:10:29.561731 1373 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:29.562180 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.704s (SENT) I0507 20:10:29.562239 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:29.562307 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:29.562434 1373 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:29.562579 1373 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.749s (election complete) I0507 20:10:29.562640 1373 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:29.563483 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:29.735714 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:29.735896 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.904s (starting election) I0507 20:10:29.735990 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:29.736501 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:29.736660 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:29.737622 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:29.962724 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:29.962865 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.925s (starting election) I0507 20:10:29.962988 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:29.963449 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:29.963616 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:29.963933 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.616s (SENT) I0507 20:10:29.964018 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:29.964246 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.451s (election complete) I0507 20:10:29.964327 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:29.964473 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:30.134534 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.570s (SENT) I0507 20:10:30.134642 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:30.134973 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.779s (election complete) I0507 20:10:30.135061 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:30.415993 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:30.416118 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.105s (starting election) I0507 20:10:30.416216 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:30.416718 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:30.416929 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:30.417979 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:30.521755 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:30.521874 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.454s (starting election) I0507 20:10:30.521967 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:30.522400 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:30.522517 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.105s (SENT) I0507 20:10:30.522583 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:30.522596 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:30.522794 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.363s (election complete) I0507 20:10:30.522845 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:30.523620 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:30.641321 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:30.641453 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.876s (starting election) I0507 20:10:30.641561 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:30.642043 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:30.642223 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:30.642477 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.904s (SENT) I0507 20:10:30.642562 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:30.642772 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.546s (election complete) I0507 20:10:30.642853 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:30.643302 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:30.886229 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:30.886353 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.249s (starting election) I0507 20:10:30.886451 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:30.886916 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:30.887109 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:30.888000 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:30.890774 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.925s (SENT) I0507 20:10:30.890833 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:30.891074 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.869s (election complete) I0507 20:10:30.891156 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:30.914669 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:30.914789 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.947s (starting election) I0507 20:10:30.914901 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:30.915331 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:30.915505 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:30.916426 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:30.977166 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.454s (SENT) I0507 20:10:30.977268 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:30.977548 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.338s (election complete) I0507 20:10:30.977643 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:31.136683 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.249s (SENT) I0507 20:10:31.136787 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:31.137105 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.664s (election complete) I0507 20:10:31.137210 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:31.189649 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:31.189791 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.936s (starting election) I0507 20:10:31.189895 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:31.190366 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:31.190554 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:31.191649 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:31.519273 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.876s (SENT) I0507 20:10:31.519404 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:31.519835 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.361s (election complete) I0507 20:10:31.519942 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:31.587534 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:28.590471 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:28.590655 (+ 184us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:28.590666 (+ 11us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:28.590743 (+ 77us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:31.587212 (+2996469us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":57} W0507 20:10:31.587857 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:10:31.802222 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:31.802354 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.131s (starting election) I0507 20:10:31.802464 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:31.802963 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:31.803162 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:31.804402 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:31.863186 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:31.863317 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.088s (starting election) I0507 20:10:31.863420 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:31.863935 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:31.864116 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:31.864228 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.947s (SENT) I0507 20:10:31.864328 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:31.864580 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.825s (election complete) I0507 20:10:31.864648 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:31.864850 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:31.881767 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:31.881893 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.340s (starting election) I0507 20:10:31.881999 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:31.882483 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:31.882694 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:31.883677 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:31.934497 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.131s (SENT) I0507 20:10:31.934604 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:31.934665 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:31.934775 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.642s (starting election) I0507 20:10:31.934909 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:31.935349 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:31.935353 1380 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.247s (election complete) I0507 20:10:31.935410 1380 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:31.935520 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:31.936375 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:31.954001 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.088s (SENT) I0507 20:10:31.954111 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:31.954412 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.338s (election complete) I0507 20:10:31.954512 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:32.127338 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.936s (SENT) I0507 20:10:32.127476 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:32.127846 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.197s (election complete) I0507 20:10:32.127955 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:32.182997 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:32.183125 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.542s (starting election) I0507 20:10:32.183236 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:32.183707 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:32.183909 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:32.184808 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:32.223141 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.340s (SENT) I0507 20:10:32.223242 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:32.223615 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.494s (election complete) I0507 20:10:32.223732 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:32.293226 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:32.293346 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.896s (starting election) I0507 20:10:32.293445 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:32.293920 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:32.294119 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:32.294939 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:32.578500 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.642s (SENT) I0507 20:10:32.578618 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:32.578932 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.518s (election complete) I0507 20:10:32.579010 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:32.718828 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:32.718957 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.199s (starting election) I0507 20:10:32.719045 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:32.719493 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:32.719693 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:32.720439 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:32.727869 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.542s (SENT) I0507 20:10:32.727959 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:32.728202 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.659s (election complete) I0507 20:10:32.728293 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:32.919152 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) W0507 20:10:32.919215 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.199s (SENT) I0507 20:10:32.919289 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.950s (starting election) I0507 20:10:32.919328 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:32.919396 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:32.919919 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:32.919922 1382 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.795s (election complete) I0507 20:10:32.919989 1382 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:32.920104 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:32.920953 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:33.190204 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:33.190361 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.666s (starting election) I0507 20:10:33.190467 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:33.191038 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:33.191278 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:33.191669 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.896s (SENT) I0507 20:10:33.191787 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:33.192093 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.334s (election complete) I0507 20:10:33.192195 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:33.192203 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:33.388020 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:33.388171 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.901s (starting election) I0507 20:10:33.388262 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:33.388715 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:33.388864 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:33.389606 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:33.526556 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:33.526706 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.554s (starting election) I0507 20:10:33.526844 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:33.527456 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:33.527658 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:33.528326 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:33.715662 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:33.715819 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.761s (starting election) I0507 20:10:33.715940 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:33.716552 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:33.716791 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:33.717823 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:33.858853 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.666s (SENT) I0507 20:10:33.859002 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:33.859494 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.637s (election complete) I0507 20:10:33.859640 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:33.871363 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.950s (SENT) I0507 20:10:33.871491 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:33.871898 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.267s (election complete) I0507 20:10:33.872067 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:34.083230 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.554s (SENT) I0507 20:10:34.083355 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:34.083678 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.196s (election complete) I0507 20:10:34.083753 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:34.139286 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:34.139412 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.565s (starting election) I0507 20:10:34.139549 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:34.139945 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:34.140076 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:34.140882 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:34.281050 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:34.281180 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.277s (starting election) I0507 20:10:34.281294 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:34.281841 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:34.282039 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:34.282860 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:34.289558 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:34.289675 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.216s (starting election) I0507 20:10:34.289779 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:34.290227 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:34.290387 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:34.290688 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.901s (SENT) I0507 20:10:34.290757 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:34.290933 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.100s (election complete) I0507 20:10:34.291002 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:34.291338 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:34.391309 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:34.391418 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.660s (starting election) I0507 20:10:34.391511 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:34.391934 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:34.392088 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:34.393133 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:34.479329 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.761s (SENT) I0507 20:10:34.479437 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:34.479779 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.138s (election complete) I0507 20:10:34.479892 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:34.507606 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.216s (SENT) I0507 20:10:34.507714 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:34.508015 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.623s (election complete) I0507 20:10:34.508095 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:34.558815 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:34.558940 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.301s (starting election) I0507 20:10:34.559042 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:34.559535 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:34.559716 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:34.560024 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.277s (SENT) I0507 20:10:34.560096 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:34.560312 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.485s (election complete) I0507 20:10:34.560375 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:34.560811 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:34.618489 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:34.618628 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.230s (starting election) I0507 20:10:34.618732 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:34.619222 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:34.619411 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:34.620434 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:34.706624 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.565s (SENT) I0507 20:10:34.706734 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:34.707123 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.479s (election complete) I0507 20:10:34.707211 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:34.850577 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.230s (SENT) I0507 20:10:34.850692 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:34.851037 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.809s (election complete) I0507 20:10:34.851124 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:34.861820 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.301s (SENT) I0507 20:10:34.861917 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:34.862207 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.527s (election complete) I0507 20:10:34.862300 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:35.053614 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.660s (SENT) I0507 20:10:35.053716 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:35.053973 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.862s (election complete) I0507 20:10:35.054060 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:35.379537 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:32.378546 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:32.378796 (+ 250us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:32.378810 (+ 14us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:32.378920 (+ 110us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:35.379216 (+3000296us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":99} W0507 20:10:35.379844 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 I0507 20:10:35.389852 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:35.389981 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.886s (starting election) I0507 20:10:35.390071 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:35.390549 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:35.390725 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:35.391971 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:35.660933 1377 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:35.661041 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.855s (starting election) I0507 20:10:35.661132 1377 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:35.661638 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:35.661800 1377 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:35.662847 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:35.916973 1377 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:35.917089 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.790s (starting election) I0507 20:10:35.917181 1377 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:35.917631 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:35.917790 1377 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:35.918702 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:36.277122 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.886s (SENT) I0507 20:10:36.277206 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:36.277346 1377 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:36.277475 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.360s (starting election) I0507 20:10:36.277575 1377 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:36.278062 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:36.278246 1377 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:36.278347 1377 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.437s (election complete) I0507 20:10:36.278383 1377 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:36.279244 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:36.517228 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.855s (SENT) I0507 20:10:36.517235 1386 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:36.517324 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:36.517357 1386 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.668s (starting election) I0507 20:10:36.517467 1386 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:36.517974 1386 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:36.518004 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.460s (election complete) I0507 20:10:36.518064 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:36.518169 1386 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:36.518935 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:36.639642 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.360s (SENT) I0507 20:10:36.639737 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:36.640051 1386 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.408s (election complete) I0507 20:10:36.640161 1386 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:36.708372 1386 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:36.708513 1386 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.647s (starting election) I0507 20:10:36.708608 1386 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } W0507 20:10:36.708984 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.790s (SENT) I0507 20:10:36.709062 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:36.709158 1386 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:36.709296 1377 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.127s (election complete) I0507 20:10:36.709373 1377 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:36.709390 1386 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:36.710255 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:36.837806 1386 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:36.837968 1386 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.589s (starting election) I0507 20:10:36.838105 1386 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:36.838779 1386 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:36.839049 1386 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:36.839984 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:36.978864 1386 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:36.979029 1386 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.130s (starting election) I0507 20:10:36.979168 1386 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:36.979691 1386 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:36.979877 1386 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:36.980880 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:37.049116 1386 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:37.049262 1386 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.848s (starting election) I0507 20:10:37.049361 1386 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:37.049857 1386 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:37.050029 1386 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:37.050647 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:37.110409 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.130s (SENT) I0507 20:10:37.110407 1386 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:37.110494 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:37.110522 1386 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.588s (starting election) I0507 20:10:37.110627 1386 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:37.111137 1386 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:37.111183 1377 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.482s (election complete) I0507 20:10:37.111239 1377 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:37.111321 1386 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:37.112056 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:37.186728 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.668s (SENT) I0507 20:10:37.186825 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:37.187188 1386 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.813s (election complete) I0507 20:10:37.187310 1386 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:37.358469 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.647s (SENT) I0507 20:10:37.358563 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:37.358996 1386 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.685s (election complete) I0507 20:10:37.359105 1386 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:37.429389 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.589s (SENT) I0507 20:10:37.429491 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:37.429924 1386 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.573s (election complete) I0507 20:10:37.430037 1386 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:37.700773 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.588s (SENT) I0507 20:10:37.700870 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:37.701282 1386 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.829s (election complete) I0507 20:10:37.701423 1386 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:37.898358 1386 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:37.898524 1386 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.512s (starting election) I0507 20:10:37.898669 1386 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:37.899425 1386 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:37.899500 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.848s (SENT) I0507 20:10:37.899581 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:37.899677 1386 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:37.899842 1386 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.385s (election complete) I0507 20:10:37.899890 1386 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:37.900828 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:38.004349 1387 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:38.004467 1387 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.141s (starting election) I0507 20:10:38.004551 1387 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:38.005025 1387 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:38.005195 1387 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:38.005908 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:38.146841 1387 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:38.146975 1387 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.175s (starting election) W0507 20:10:38.146996 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.141s (SENT) I0507 20:10:38.147080 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:38.147079 1387 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:38.147517 1387 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:38.147559 1386 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.949s (election complete) I0507 20:10:38.147666 1386 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:38.148128 1387 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:38.148357 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:38.285771 1387 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:38.285884 1387 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.975s (starting election) I0507 20:10:38.285961 1387 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:38.286465 1387 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:38.286630 1387 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:38.287338 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:38.323742 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.175s (SENT) I0507 20:10:38.323897 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:38.324281 1387 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.436s (election complete) I0507 20:10:38.324375 1387 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:38.412917 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.512s (SENT) I0507 20:10:38.413051 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:38.413450 1387 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.564s (election complete) I0507 20:10:38.413565 1387 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:38.531575 1387 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:38.531683 1387 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.633s (starting election) I0507 20:10:38.531757 1387 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:38.532212 1387 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:38.532371 1387 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:38.533198 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:38.760979 1387 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:38.761078 1387 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.377s (starting election) I0507 20:10:38.761147 1387 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:38.761566 1387 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:38.761699 1387 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:38.762346 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:38.978816 1387 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:38.978947 1387 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.106s (starting election) I0507 20:10:38.979023 1387 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:38.979470 1387 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:38.979650 1387 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:38.980325 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:39.085682 1387 raft_consensus.cc:435] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:39.085805 1387 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.980s (starting election) I0507 20:10:39.085913 1387 raft_consensus.cc:457] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:39.086378 1387 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:39.086551 1387 leader_election.cc:230] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:39.086998 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.106s (SENT) I0507 20:10:39.087164 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. W0507 20:10:39.087385 296 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:39.087487 1387 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.667s (election complete) I0507 20:10:39.087534 1387 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:39.139168 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.377s (SENT) I0507 20:10:39.139328 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:39.139423 1387 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:39.139520 1387 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.101s (starting election) I0507 20:10:39.139596 1387 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:39.140038 1387 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:39.140203 1387 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:39.140293 1387 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.184s (election complete) I0507 20:10:39.140327 1387 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:39.140895 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:39.165484 1389 raft_consensus.cc:435] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:39.165598 1389 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.853s (starting election) I0507 20:10:39.165699 1389 raft_consensus.cc:457] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } I0507 20:10:39.166122 1389 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 I0507 20:10:39.166318 1389 leader_election.cc:230] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa W0507 20:10:39.166895 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.633s (SENT) I0507 20:10:39.166954 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:10:39.167076 296 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) I0507 20:10:39.167102 1389 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.685s (election complete) I0507 20:10:39.167136 1389 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:39.179502 660 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:36.182479 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.195:38453 0507 20:10:36.182694 (+ 215us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:36.182707 (+ 13us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:36.182797 (+ 90us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:39.179166 (+2996369us) negotiation.cc:304] Negotiation complete: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 Metrics: {"client-negotiator.queue_time_us":63} W0507 20:10:39.179760 659 meta_cache.cc:216] Tablet a0376a5e3b9f495aa00d366b35594a27: Replica b5e887ddd8a14cf3b2c24a0aa14be025 (127.0.52.195:38453) has failed: Timed out: Client connection negotiation failed: client connection to 127.0.52.195:38453 W0507 20:10:39.242803 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.101s (SENT) I0507 20:10:39.242909 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:39.243211 1389 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.097s (election complete) I0507 20:10:39.243288 1389 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:39.263934 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Timed out: RequestConsensusVote RPC to 127.0.52.195:38453 timed out after 0.975s (SENT) I0507 20:10:39.264031 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:39.264344 1389 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.720s (election complete) I0507 20:10:39.264427 1389 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority I0507 20:10:39.340955 1389 raft_consensus.cc:435] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election (detected failure of leader b5e887ddd8a14cf3b2c24a0aa14be025) I0507 20:10:39.341055 1389 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.311s (starting election) I0507 20:10:39.341125 1389 raft_consensus.cc:457] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } I0507 20:10:39.341584 1389 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer 5014824b94b64fd49da2c1a645c078aa I0507 20:10:39.341753 1389 leader_election.cc:230] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Requesting pre-vote from peer b5e887ddd8a14cf3b2c24a0aa14be025 W0507 20:10:39.342707 296 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer 5014824b94b64fd49da2c1a645c078aa: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111) W0507 20:10:39.443085 659 batcher.cc:348] Timed out: Failed to write batch of 1 ops to tablet a0376a5e3b9f495aa00d366b35594a27 after 261 attempt(s): Failed to write to server: (no server available): Write(tablet: a0376a5e3b9f495aa00d366b35594a27, num_ops: 1, num_attempts: 261) passed its deadline: Illegal state: Replica d1bf0f72a88f4483a1906bc8246849dd is not leader of this config. Role: FOLLOWER. Consensus state: current_term: 3 leader_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b5e887ddd8a14cf3b2c24a0aa14be025" member_type: VOTER last_known_addr { host: "127.0.52.195" port: 38453 } } peers { permanent_uuid: "d1bf0f72a88f4483a1906bc8246849dd" member_type: VOTER last_known_addr { host: "127.0.52.193" port: 46057 } } peers { permanent_uuid: "5014824b94b64fd49da2c1a645c078aa" member_type: VOTER last_known_addr { host: "127.0.52.194" port: 43942 } } } /data/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/client-negotiation-failover-itest.cc:186: Failure Failed Bad status: IO error: Some errors occurred W0507 20:10:39.446007 541 connection.cc:218] RPC call timeout handler was delayed by 273.53s! This may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow an additional 3s for a response. W0507 20:10:39.446118 541 connection.cc:218] RPC call timeout handler was delayed by 273.452s! This may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow an additional 3s for a response. W0507 20:10:39.446144 541 connection.cc:218] RPC call timeout handler was delayed by 273.448s! This may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow an additional 3s for a response. I0507 20:10:39.446245 211 external_mini_cluster.cc:1019] Attempting to check leaks for /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver pid 277 W0507 20:10:39.446271 541 connection.cc:512] client connection to 127.0.52.194:43942 recv error: Network error: failed to read from TLS socket (remote: 127.0.52.194:43942): Connection reset by peer (error 104) W0507 20:10:39.446388 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: failed to read from TLS socket (remote: 127.0.52.194:43942): Connection reset by peer (error 104). Retrying in the next heartbeat period. Already tried 1 times. W0507 20:10:39.446471 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: failed to read from TLS socket (remote: 127.0.52.194:43942): Connection reset by peer (error 104). Retrying in the next heartbeat period. Already tried 1 times. W0507 20:10:39.446516 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: failed to read from TLS socket (remote: 127.0.52.194:43942): Connection reset by peer (error 104). Retrying in the next heartbeat period. Already tried 1 times. W0507 20:10:39.447479 542 connection.cc:218] RPC call timeout handler was delayed by 273.441s! This may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow an additional 3s for a response. W0507 20:10:39.447515 542 connection.cc:218] RPC call timeout handler was delayed by 273.44s! This may be due to a process-wide pause such as swapping, logging-related delays, or allocator lock contention. Will allow an additional 3s for a response. I0507 20:10:39.449602 869 consensus_queue.cc:543] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Leader has been unable to successfully communicate with peer d1bf0f72a88f4483a1906bc8246849dd for more than 300 seconds (300.460s) W0507 20:10:39.453825 297 leader_election.cc:286] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Cannot send after transport endpoint shutdown (error 108) I0507 20:10:39.455426 297 leader_election.cc:257] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. W0507 20:10:39.455972 297 leader_election.cc:286] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Cannot send after transport endpoint shutdown (error 108) I0507 20:10:39.456638 297 leader_election.cc:257] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 4 pre-election: Election decided. Result: candidate lost. I0507 20:10:39.456153 1389 raft_consensus.cc:2704] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.340s (election complete) I0507 20:10:39.457212 1389 raft_consensus.cc:2459] T 7161aaf1a71d455883ef478337dfaf19 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority I0507 20:10:39.457334 1387 raft_consensus.cc:2704] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.742s (election complete) I0507 20:10:39.457372 1387 raft_consensus.cc:2459] T a0376a5e3b9f495aa00d366b35594a27 P d1bf0f72a88f4483a1906bc8246849dd [term 3 FOLLOWER]: Leader pre-election lost for term 4. Reason: could not achieve majority W0507 20:10:39.457564 297 leader_election.cc:286] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: RPC error from VoteRequest() call to peer b5e887ddd8a14cf3b2c24a0aa14be025: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Cannot send after transport endpoint shutdown (error 108) I0507 20:10:39.457643 297 leader_election.cc:257] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [CANDIDATE]: Term 5 pre-election: Election decided. Result: candidate lost. I0507 20:10:39.458133 1389 raft_consensus.cc:2704] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd: Snoozing failure detection for 0.621s (election complete) I0507 20:10:39.458209 1389 raft_consensus.cc:2459] T d80f4fae4b9f4d238f7f626507290390 P d1bf0f72a88f4483a1906bc8246849dd [term 4 FOLLOWER]: Leader pre-election lost for term 5. Reason: could not achieve majority W0507 20:10:39.460340 297 connection.cc:185] Error closing socket: Network error: TlsSocket::Close: Connection reset by peer I0507 20:10:39.463973 652 maintenance_manager.cc:304] P b5e887ddd8a14cf3b2c24a0aa14be025: Scheduling FlushMRSOp(7161aaf1a71d455883ef478337dfaf19): perf score=0.089909 I0507 20:10:39.465219 546 tablet.cc:1152] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering stage 1 (old memrowset already frozen for inserts) I0507 20:10:39.465267 546 compaction.cc:913] Selected 1 rowsets to compact: I0507 20:10:39.465332 546 compaction.cc:916] memrowset(current size on disk: ~0 bytes) I0507 20:10:39.465361 546 tablet.cc:1154] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Memstore in-memory size: 763 bytes I0507 20:10:39.465421 546 tablet.cc:1443] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6249363615459508224 or (T in {6249363615459508224})}] W0507 20:10:39.468544 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.452062 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43136 0507 20:10:39.467335 (+ 15273us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.467339 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.467359 (+ 20us) server_negotiation.cc:373] Connection header received 0507 20:10:39.467421 (+ 62us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.467425 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.467510 (+ 85us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.467751 (+ 241us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43136: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":15117,"thread_start_us":15026,"threads_started":1} W0507 20:10:39.469053 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.452062 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43148 0507 20:10:39.468695 (+ 16633us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.468698 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.468709 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.468745 (+ 36us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.468748 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.468796 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.468894 (+ 98us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43148: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"mutex_wait_us":84,"server-negotiator.queue_time_us":16556,"threads_started":1} W0507 20:10:39.469488 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.453163 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43132 0507 20:10:39.469167 (+ 16004us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.469171 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.469181 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.469210 (+ 29us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.469212 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.469254 (+ 42us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.469342 (+ 88us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43132: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":15924,"threads_started":1} W0507 20:10:39.469959 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.459814 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43130 0507 20:10:39.469608 (+ 9794us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.469610 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.469626 (+ 16us) server_negotiation.cc:373] Connection header received 0507 20:10:39.469651 (+ 25us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.469654 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.469695 (+ 41us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.469808 (+ 113us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43130: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":9707,"threads_started":1} W0507 20:10:39.470399 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.467215 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43144 0507 20:10:39.470060 (+ 2845us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.470062 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.470071 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.470096 (+ 25us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.470099 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.470148 (+ 49us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.470248 (+ 100us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43144: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":2779,"threads_started":1} I0507 20:10:39.483893 403 generic_service.cc:169] Checking for leaks (request via RPC) W0507 20:10:39.483903 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.470664 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43165 0507 20:10:39.482327 (+ 11663us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.482330 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.482339 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.482366 (+ 27us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.482369 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.482414 (+ 45us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.483430 (+ 1016us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43165: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":72} W0507 20:10:39.484459 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.470851 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43262 0507 20:10:39.484066 (+ 13215us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.484069 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.484079 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.484115 (+ 36us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.484117 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.484180 (+ 63us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.484277 (+ 97us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43262: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":13147,"thread_start_us":1523,"threads_started":1} W0507 20:10:39.485723 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.470830 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43225 0507 20:10:39.483120 (+ 12290us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.483123 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.483137 (+ 14us) server_negotiation.cc:373] Connection header received 0507 20:10:39.483165 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.483168 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.483215 (+ 47us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.483871 (+ 656us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43225: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":12158} W0507 20:10:39.486351 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.485466 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43169 0507 20:10:39.485925 (+ 459us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.485928 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.485939 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.485968 (+ 29us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.485970 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.486034 (+ 64us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.486147 (+ 113us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43169: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":388} W0507 20:10:39.488020 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.470773 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43201 0507 20:10:39.482519 (+ 11746us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.482522 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.482529 (+ 7us) server_negotiation.cc:373] Connection header received 0507 20:10:39.482552 (+ 23us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.482554 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.482592 (+ 38us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.483820 (+ 1228us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43201: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":4182} W0507 20:10:39.488243 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.470996 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43157 0507 20:10:39.484701 (+ 13705us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.484704 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.484714 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.484742 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.484744 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.484790 (+ 46us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.484929 (+ 139us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43157: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":13643,"thread_start_us":14348,"threads_started":1} W0507 20:10:39.489468 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.488801 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43185 0507 20:10:39.488935 (+ 134us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.488938 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.488948 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.488976 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.488979 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.489027 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.489130 (+ 103us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43185: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":53} W0507 20:10:39.489616 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.485546 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43181 0507 20:10:39.486545 (+ 999us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.486548 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.486559 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.486587 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.486589 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.486636 (+ 47us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.487119 (+ 483us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43181: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":922} W0507 20:10:39.490473 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.489761 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43213 0507 20:10:39.490048 (+ 287us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.490050 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.490058 (+ 8us) server_negotiation.cc:373] Connection header received 0507 20:10:39.490080 (+ 22us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.490082 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.490129 (+ 47us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.490223 (+ 94us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43213: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":232} W0507 20:10:39.491219 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.489699 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43189 0507 20:10:39.489844 (+ 145us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.489847 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.489858 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.489887 (+ 29us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.489890 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.490920 (+ 1030us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.491003 (+ 83us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43189: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":76} W0507 20:10:39.491969 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.490358 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43242 0507 20:10:39.491385 (+ 1027us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.491387 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.491397 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.491425 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.491428 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.491473 (+ 45us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.491564 (+ 91us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43242: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":969} W0507 20:10:39.492763 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.491080 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43250 0507 20:10:39.492156 (+ 1076us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.492159 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.492170 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.492197 (+ 27us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.492200 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.492246 (+ 46us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.492337 (+ 91us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43250: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":985} W0507 20:10:39.499461 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.491699 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43274 0507 20:10:39.492937 (+ 1238us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.492940 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.492949 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.492976 (+ 27us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.492978 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.493024 (+ 46us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.493141 (+ 117us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43274: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":1181} W0507 20:10:39.500169 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.490311 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43217 0507 20:10:39.490636 (+ 325us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.490639 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.490649 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.490676 (+ 27us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.490679 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.490740 (+ 61us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.490835 (+ 95us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43217: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":260} W0507 20:10:39.500850 1400 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.470717 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43173 0507 20:10:39.478203 (+ 7486us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.478208 (+ 5us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.481899 (+ 3691us) server_negotiation.cc:373] Connection header received 0507 20:10:39.481938 (+ 39us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.481941 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.482006 (+ 65us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.483794 (+ 1788us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43173: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":1723} W0507 20:10:39.502053 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.492499 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43302 0507 20:10:39.500352 (+ 7853us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.500355 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.500365 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.500397 (+ 32us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.500400 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.500445 (+ 45us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.500679 (+ 234us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43302: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":7793,"threads_started":1} W0507 20:10:39.502792 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.492464 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43286 0507 20:10:39.499888 (+ 7424us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.499892 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.499906 (+ 14us) server_negotiation.cc:373] Connection header received 0507 20:10:39.499945 (+ 39us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.499948 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.500008 (+ 60us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.500124 (+ 116us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43286: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":7336} W0507 20:10:39.503867 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.470807 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43221 0507 20:10:39.480012 (+ 9205us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.480015 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.480032 (+ 17us) server_negotiation.cc:373] Connection header received 0507 20:10:39.480089 (+ 57us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.480093 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.480190 (+ 97us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.482239 (+ 2049us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43221: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":9130} W0507 20:10:39.504645 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.491644 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43266 0507 20:10:39.492580 (+ 936us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.492583 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.492592 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.493322 (+ 730us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.493325 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.493373 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.493464 (+ 91us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43266: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":874} W0507 20:10:39.507338 1400 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.493716 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43294 0507 20:10:39.501043 (+ 7327us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.501046 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.501057 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.501086 (+ 29us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.501089 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.501158 (+ 69us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.501255 (+ 97us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43294: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":7244} W0507 20:10:39.508456 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.502083 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43161 0507 20:10:39.503561 (+ 1478us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.503564 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.503573 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.503598 (+ 25us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.503600 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.503665 (+ 65us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.503758 (+ 93us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43161: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":921} W0507 20:10:39.509922 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.501988 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43140 0507 20:10:39.502237 (+ 249us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.502240 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.502250 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.502278 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.502281 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.502326 (+ 45us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.502580 (+ 254us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43140: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":177} W0507 20:10:39.511441 1400 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.503180 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43355 0507 20:10:39.507512 (+ 4332us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.507516 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.507527 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.507557 (+ 30us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.507560 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.507608 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.507718 (+ 110us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43355: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":4269,"threads_started":1} W0507 20:10:39.511881 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.504328 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43327 0507 20:10:39.508643 (+ 4315us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.508646 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.508657 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.508690 (+ 33us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.508694 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.508757 (+ 63us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.509171 (+ 414us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43327: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":4227} W0507 20:10:39.513133 1406 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.493760 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43318 0507 20:10:39.501484 (+ 7724us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.501487 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.501502 (+ 15us) server_negotiation.cc:373] Connection header received 0507 20:10:39.501528 (+ 26us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.501531 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.501575 (+ 44us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.504210 (+ 2635us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43318: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":7632,"thread_start_us":8700,"threads_started":1} W0507 20:10:39.513586 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.502107 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43205 0507 20:10:39.504009 (+ 1902us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.504012 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.504023 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.504055 (+ 32us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.504057 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.504112 (+ 55us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.504211 (+ 99us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43205: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":1832,"threads_started":1} W0507 20:10:39.513929 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.502540 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43322 0507 20:10:39.504768 (+ 2228us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.504771 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.504781 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.504810 (+ 29us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.504812 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.504858 (+ 46us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.505256 (+ 398us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43322: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":2161} W0507 20:10:39.514245 1401 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.472432 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43278 0507 20:10:39.493822 (+ 21390us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.493825 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.493890 (+ 65us) server_negotiation.cc:373] Connection header received 0507 20:10:39.493920 (+ 30us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.493923 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.493972 (+ 49us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.501905 (+ 7933us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43278: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":13129,"thread_start_us":19958,"threads_started":1} W0507 20:10:39.515024 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.507990 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43197 0507 20:10:39.512077 (+ 4087us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.512081 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.512094 (+ 13us) server_negotiation.cc:373] Connection header received 0507 20:10:39.512128 (+ 34us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.512133 (+ 5us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.512192 (+ 59us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.512307 (+ 115us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43197: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":4001} W0507 20:10:39.516054 1400 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.508037 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43233 0507 20:10:39.512555 (+ 4518us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.512559 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.512570 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.512600 (+ 30us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.512604 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.512665 (+ 61us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.513057 (+ 392us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43233: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":4440,"threads_started":1} W0507 20:10:39.517186 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.504388 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43347 0507 20:10:39.510131 (+ 5743us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.510135 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.510146 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.510179 (+ 33us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.510183 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.510848 (+ 665us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.510985 (+ 137us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43347: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":5664,"threads_started":1} W0507 20:10:39.517436 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.515865 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43209 0507 20:10:39.516037 (+ 172us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.516040 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.516052 (+ 12us) server_negotiation.cc:373] Connection header received 0507 20:10:39.516082 (+ 30us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.516085 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.516726 (+ 641us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.516839 (+ 113us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43209: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":52} W0507 20:10:39.517891 1404 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.491728 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43282 0507 20:10:39.502457 (+ 10729us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.502459 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.503254 (+ 795us) server_negotiation.cc:373] Connection header received 0507 20:10:39.503289 (+ 35us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.503292 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.503346 (+ 54us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.514409 (+ 11063us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43282: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":8046,"thread_start_us":1863,"threads_started":1} W0507 20:10:39.519809 1404 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.518067 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43367 0507 20:10:39.518229 (+ 162us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.518234 (+ 5us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.518247 (+ 13us) server_negotiation.cc:373] Connection header received 0507 20:10:39.518284 (+ 37us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.518288 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.518372 (+ 84us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.518490 (+ 118us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43367: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":53} W0507 20:10:39.521281 1397 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.502044 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43152 0507 20:10:39.502844 (+ 800us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.502847 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.502857 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.502909 (+ 52us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.502912 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.502960 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.511689 (+ 8729us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43152: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":705} W0507 20:10:39.524929 1400 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.518578 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43396 0507 20:10:39.518736 (+ 158us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.518740 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.518751 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.518821 (+ 70us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.518825 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.519457 (+ 632us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.519579 (+ 122us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43396: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":78} W0507 20:10:39.526448 1404 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.521671 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43246 0507 20:10:39.522185 (+ 514us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.522192 (+ 7us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.522204 (+ 12us) server_negotiation.cc:373] Connection header received 0507 20:10:39.522237 (+ 33us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.522240 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.522292 (+ 52us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.522401 (+ 109us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43246: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":421} W0507 20:10:39.526983 1406 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.522163 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43530 0507 20:10:39.523032 (+ 869us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.523037 (+ 5us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.523049 (+ 12us) server_negotiation.cc:373] Connection header received 0507 20:10:39.523085 (+ 36us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.523089 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.523173 (+ 84us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.523295 (+ 122us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43530: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":782} W0507 20:10:39.528234 1406 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.523007 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43310 0507 20:10:39.527110 (+ 4103us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.527113 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.527124 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.527152 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.527155 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.527203 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.527295 (+ 92us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43310: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":4032} W0507 20:10:39.529515 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.522903 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43534 0507 20:10:39.524181 (+ 1278us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.524184 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.524193 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.524219 (+ 26us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.524221 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.524266 (+ 45us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.524361 (+ 95us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43534: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":1180} W0507 20:10:39.530023 1401 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.522054 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43453 0507 20:10:39.522499 (+ 445us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.522504 (+ 5us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.522517 (+ 13us) server_negotiation.cc:373] Connection header received 0507 20:10:39.522552 (+ 35us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.522556 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.523557 (+ 1001us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.524822 (+ 1265us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43453: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":357} W0507 20:10:39.530407 1400 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.522921 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43270 0507 20:10:39.525089 (+ 2168us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.525093 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.525115 (+ 22us) server_negotiation.cc:373] Connection header received 0507 20:10:39.525154 (+ 39us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.525158 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.525216 (+ 58us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.525332 (+ 116us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43270: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":2088} W0507 20:10:39.530519 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.518639 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43413 0507 20:10:39.519011 (+ 372us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.519013 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.519022 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.519045 (+ 23us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.519048 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.519089 (+ 41us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.519181 (+ 92us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43413: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":312} W0507 20:10:39.531129 1404 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.522966 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43290 0507 20:10:39.526563 (+ 3597us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.526566 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.526576 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.526604 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.526607 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.526653 (+ 46us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.526748 (+ 95us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43290: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":3536} W0507 20:10:39.531888 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.522821 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43254 0507 20:10:39.523790 (+ 969us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.523793 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.523802 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.523829 (+ 27us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.523832 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.523875 (+ 43us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.523975 (+ 100us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43254: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":311} W0507 20:10:39.535986 1406 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.523373 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43229 0507 20:10:39.529034 (+ 5661us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.529037 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.529047 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.529076 (+ 29us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.529079 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.529128 (+ 49us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.529224 (+ 96us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43229: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":2846} W0507 20:10:39.537984 1406 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.535800 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43509 0507 20:10:39.536096 (+ 296us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.536099 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.536109 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.536137 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.536139 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.536199 (+ 60us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.536296 (+ 97us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43509: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":233} W0507 20:10:39.544853 1405 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.503136 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43343 0507 20:10:39.509521 (+ 6385us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.509525 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.509719 (+ 194us) server_negotiation.cc:373] Connection header received 0507 20:10:39.509757 (+ 38us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.509761 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.509823 (+ 62us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.516563 (+ 6740us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43343: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":3707} W0507 20:10:39.547601 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.518612 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43405 0507 20:10:39.518730 (+ 118us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.518733 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.518743 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.518773 (+ 30us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.518775 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.518823 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.527813 (+ 8990us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43405: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":50} W0507 20:10:39.548454 1404 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.535312 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43445 0507 20:10:39.535469 (+ 157us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.535472 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.535481 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.535504 (+ 23us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.535506 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.536698 (+ 1192us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.536881 (+ 183us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43445: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":88} W0507 20:10:39.548715 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.527635 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43359 0507 20:10:39.529640 (+ 2005us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.529643 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.529653 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.529690 (+ 37us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.529693 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.529737 (+ 44us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.529830 (+ 93us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43359: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":1926} W0507 20:10:39.550104 1401 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.530276 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43363 0507 20:10:39.531461 (+ 1185us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.531464 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.531474 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.531506 (+ 32us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.531508 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.531554 (+ 46us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.531657 (+ 103us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43363: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":1116} W0507 20:10:39.550990 1400 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.535554 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43497 0507 20:10:39.535633 (+ 79us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.535635 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.535642 (+ 7us) server_negotiation.cc:373] Connection header received 0507 20:10:39.535662 (+ 20us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.535665 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.535706 (+ 41us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.536465 (+ 759us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43497: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":17} W0507 20:10:39.551434 1401 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.547203 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43306 0507 20:10:39.550242 (+ 3039us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.550246 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.550257 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.550288 (+ 31us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.550291 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.550342 (+ 51us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.550442 (+ 100us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43306: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":2975,"threads_started":1} W0507 20:10:39.552021 1394 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.534662 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43384 0507 20:10:39.534985 (+ 323us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.534988 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.535000 (+ 12us) server_negotiation.cc:373] Connection header received 0507 20:10:39.535029 (+ 29us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.535032 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.535079 (+ 47us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.537322 (+ 2243us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43384: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":21} W0507 20:10:39.552402 1402 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.534748 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43400 0507 20:10:39.535201 (+ 453us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.535203 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.535212 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.535233 (+ 21us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.535236 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.536975 (+ 1739us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.537064 (+ 89us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43400: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":47} W0507 20:10:39.552587 1407 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.518527 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43392 0507 20:10:39.523640 (+ 5113us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.523643 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.525529 (+ 1886us) server_negotiation.cc:373] Connection header received 0507 20:10:39.525568 (+ 39us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.525572 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.525642 (+ 70us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.532840 (+ 7198us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43392: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":73} W0507 20:10:39.553014 1397 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.526189 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43237 0507 20:10:39.529346 (+ 3157us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.529349 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.529357 (+ 8us) server_negotiation.cc:373] Connection header received 0507 20:10:39.529382 (+ 25us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.529385 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.530595 (+ 1210us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.530707 (+ 112us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43237: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":3094,"thread_start_us":20768,"threads_started":1} I0507 20:10:39.553038 1392 consensus_queue.cc:543] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Leader has been unable to successfully communicate with peer 5014824b94b64fd49da2c1a645c078aa for more than 300 seconds (301.687s) I0507 20:10:39.553493 869 consensus_queue.cc:543] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Leader has been unable to successfully communicate with peer 5014824b94b64fd49da2c1a645c078aa for more than 300 seconds (300.555s) I0507 20:10:39.554041 1393 consensus_queue.cc:543] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 [LEADER]: Leader has been unable to successfully communicate with peer 5014824b94b64fd49da2c1a645c078aa for more than 300 seconds (300.559s) W0507 20:10:39.554704 1400 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.550816 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43429 0507 20:10:39.551240 (+ 424us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.551243 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.551256 (+ 13us) server_negotiation.cc:373] Connection header received 0507 20:10:39.551286 (+ 30us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.551289 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.552250 (+ 961us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.552344 (+ 94us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43429: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"mutex_wait_us":308,"server-negotiator.queue_time_us":349} W0507 20:10:39.554813 1401 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.551349 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43433 0507 20:10:39.551524 (+ 175us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.551527 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.551537 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.551570 (+ 33us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.551573 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.551638 (+ 65us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.551746 (+ 108us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43433: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":116} W0507 20:10:39.567796 1396 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.547157 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43298 0507 20:10:39.548829 (+ 1672us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.548832 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.548843 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.548872 (+ 29us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.548874 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.548922 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.549025 (+ 103us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43298: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":1605} W0507 20:10:39.568416 1404 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.544016 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43522 0507 20:10:39.549189 (+ 5173us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.549192 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.549203 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.549229 (+ 26us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.549232 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.549277 (+ 45us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.549697 (+ 420us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43522: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":4515,"threads_started":1} W0507 20:10:39.569315 1405 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.537308 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43376 0507 20:10:39.546043 (+ 8735us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.546046 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.546058 (+ 12us) server_negotiation.cc:373] Connection header received 0507 20:10:39.546096 (+ 38us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.546100 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.546634 (+ 534us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.546739 (+ 105us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43376: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":7795} W0507 20:10:39.569510 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.543919 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43517 0507 20:10:39.547910 (+ 3991us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.547914 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.547928 (+ 14us) server_negotiation.cc:373] Connection header received 0507 20:10:39.547970 (+ 42us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.547973 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.548027 (+ 54us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.549495 (+ 1468us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43517: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":3907} W0507 20:10:39.570176 1412 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.550163 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43421 0507 20:10:39.552184 (+ 2021us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.552187 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.552933 (+ 746us) server_negotiation.cc:373] Connection header received 0507 20:10:39.552969 (+ 36us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.552972 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.553035 (+ 63us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.554086 (+ 1051us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43421: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":953} W0507 20:10:39.570911 1406 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.537259 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43372 0507 20:10:39.545823 (+ 8564us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.545827 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.545844 (+ 17us) server_negotiation.cc:373] Connection header received 0507 20:10:39.545910 (+ 66us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.545913 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.545990 (+ 77us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.546705 (+ 715us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43372: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":778} W0507 20:10:39.571044 1399 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.507270 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43193 0507 20:10:39.514392 (+ 7122us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.514397 (+ 5us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.514527 (+ 130us) server_negotiation.cc:373] Connection header received 0507 20:10:39.514566 (+ 39us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.514571 (+ 5us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.514641 (+ 70us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.518414 (+ 3773us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43193: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":3698} W0507 20:10:39.571167 1408 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.522126 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43505 0507 20:10:39.527664 (+ 5538us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.527666 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.527765 (+ 99us) server_negotiation.cc:373] Connection header received 0507 20:10:39.527795 (+ 30us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.527798 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.527847 (+ 49us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.541572 (+ 13725us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43505: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":667} W0507 20:10:39.575064 1410 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.523054 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43351 0507 20:10:39.532767 (+ 9713us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.532771 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.533010 (+ 239us) server_negotiation.cc:373] Connection header received 0507 20:10:39.533041 (+ 31us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.533044 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.533092 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.552864 (+ 19772us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43351: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":6380,"thread_start_us":14105,"threads_started":1} W0507 20:10:39.575330 1411 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.537343 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43380 0507 20:10:39.548677 (+ 11334us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.548681 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.549373 (+ 692us) server_negotiation.cc:373] Connection header received 0507 20:10:39.549396 (+ 23us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.549399 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.549434 (+ 35us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.568565 (+ 19131us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43380: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":9298,"thread_start_us":12751,"threads_started":1} W0507 20:10:39.575517 1409 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.534778 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43441 0507 20:10:39.537963 (+ 3185us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.537967 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.542024 (+ 4057us) server_negotiation.cc:373] Connection header received 0507 20:10:39.542077 (+ 53us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.542082 (+ 5us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.542159 (+ 77us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.553517 (+ 11358us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43441: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":31} I0507 20:10:39.577888 546 multi_column_writer.cc:98] Opened CFile writers for 2 column(s) W0507 20:10:39.584692 1399 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.582112 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43437 0507 20:10:39.582218 (+ 106us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.582220 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.582228 (+ 8us) server_negotiation.cc:373] Connection header received 0507 20:10:39.582258 (+ 30us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.582260 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.582348 (+ 88us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.582468 (+ 120us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43437: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":42,"spinlock_wait_cycles":7936} W0507 20:10:39.585438 1411 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.581363 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43388 0507 20:10:39.581982 (+ 619us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.581986 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.582001 (+ 15us) server_negotiation.cc:373] Connection header received 0507 20:10:39.582050 (+ 49us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.582053 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.582317 (+ 264us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.582470 (+ 153us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43388: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":36} W0507 20:10:39.585542 1409 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.581298 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43339 0507 20:10:39.582163 (+ 865us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.582166 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.582172 (+ 6us) server_negotiation.cc:373] Connection header received 0507 20:10:39.582191 (+ 19us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.582193 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.582287 (+ 94us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.582419 (+ 132us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43339: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":39} W0507 20:10:39.591442 1408 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.582016 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43425 0507 20:10:39.582554 (+ 538us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.582556 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.582566 (+ 10us) server_negotiation.cc:373] Connection header received 0507 20:10:39.582594 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.582597 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.582821 (+ 224us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.582949 (+ 128us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43425: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":26} W0507 20:10:39.591903 1410 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.581399 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43417 0507 20:10:39.582017 (+ 618us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.582021 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.582036 (+ 15us) server_negotiation.cc:373] Connection header received 0507 20:10:39.582077 (+ 41us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.582080 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.582998 (+ 918us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.583124 (+ 126us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43417: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":19} W0507 20:10:39.592257 1412 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.584469 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43457 0507 20:10:39.584734 (+ 265us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.584737 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.584748 (+ 11us) server_negotiation.cc:373] Connection header received 0507 20:10:39.584776 (+ 28us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.584779 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.584842 (+ 63us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.584941 (+ 99us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43457: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":41} W0507 20:10:39.592491 1405 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.584572 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43526 0507 20:10:39.585057 (+ 485us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.585059 (+ 2us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.585068 (+ 9us) server_negotiation.cc:373] Connection header received 0507 20:10:39.585106 (+ 38us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.585109 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.585152 (+ 43us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.585246 (+ 94us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43526: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":22} W0507 20:10:39.592598 1406 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.583967 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43449 0507 20:10:39.584111 (+ 144us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.584114 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.584128 (+ 14us) server_negotiation.cc:373] Connection header received 0507 20:10:39.584163 (+ 35us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.584166 (+ 3us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.584214 (+ 48us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.584314 (+ 100us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43449: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":49} W0507 20:10:39.592828 1395 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.584536 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43501 0507 20:10:39.591032 (+ 6496us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.591036 (+ 4us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.591049 (+ 13us) server_negotiation.cc:373] Connection header received 0507 20:10:39.591082 (+ 33us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.591084 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.591144 (+ 60us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.591263 (+ 119us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43501: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":20} W0507 20:10:39.593067 1415 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.579345 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43314 0507 20:10:39.580055 (+ 710us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.580060 (+ 5us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.580083 (+ 23us) server_negotiation.cc:373] Connection header received 0507 20:10:39.580159 (+ 76us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.580163 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.580257 (+ 94us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.592371 (+ 12114us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43314: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":44} W0507 20:10:39.593245 1409 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.585740 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43538 0507 20:10:39.589357 (+ 3617us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.589360 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.589374 (+ 14us) server_negotiation.cc:373] Connection header received 0507 20:10:39.589411 (+ 37us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.589415 (+ 4us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.589469 (+ 54us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.589575 (+ 106us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43538: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":3519} W0507 20:10:39.593559 1414 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.580558 (+ 0us) reactor.cc:577] Submitting negotiation task for server connection from 127.0.0.1:43331 0507 20:10:39.583575 (+ 3017us) server_negotiation.cc:176] Beginning negotiation 0507 20:10:39.583578 (+ 3us) server_negotiation.cc:365] Waiting for connection header 0507 20:10:39.584181 (+ 603us) server_negotiation.cc:373] Connection header received 0507 20:10:39.584215 (+ 34us) server_negotiation.cc:329] Received NEGOTIATE NegotiatePB request 0507 20:10:39.584217 (+ 2us) server_negotiation.cc:412] Received NEGOTIATE request from client 0507 20:10:39.584266 (+ 49us) server_negotiation.cc:341] Sending NEGOTIATE NegotiatePB response 0507 20:10:39.593396 (+ 9130us) negotiation.cc:304] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43331: BlockingWrite error: write error: Broken pipe (error 32) Metrics: {"server-negotiator.queue_time_us":47} W0507 20:10:39.595989 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 2 times. W0507 20:10:39.596122 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 2 times. W0507 20:10:39.596330 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 2 times. W0507 20:10:39.624075 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 3 times. W0507 20:10:39.624393 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 3 times. W0507 20:10:39.624639 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 3 times. I0507 20:10:39.625814 241 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet a0376a5e3b9f495aa00d366b35594a27 (attempt 1) W0507 20:10:39.626039 241 catalog_manager.cc:3404] ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a0376a5e3b9f495aa00d366b35594a27 with cas_config_opid_index -1: no candidate replacement replica found for tablet a0376a5e3b9f495aa00d366b35594a27 (table kudu1580 [id=ece6530d2ea44ecb92898941bb6fbf76]); the total number of registered tablet servers (3) does not allow for replacement of the failed replica: at least 4 tablet servers are required; consider either adding an additional tablet server or running the cluster with --raft_prepare_replacement_before_eviction=false I0507 20:10:39.626094 241 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a0376a5e3b9f495aa00d366b35594a27 with cas_config_opid_index -1 with a delay of 32 ms (attempt = 1) I0507 20:10:39.626205 241 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet 7161aaf1a71d455883ef478337dfaf19 (attempt 1) I0507 20:10:39.626281 241 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7161aaf1a71d455883ef478337dfaf19 with cas_config_opid_index -1 with a delay of 64 ms (attempt = 1) W0507 20:10:39.628383 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 4 times. W0507 20:10:39.631436 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 4 times. W0507 20:10:39.639935 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 4 times. I0507 20:10:39.642750 546 tablet.cc:1528] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I0507 20:10:39.643273 546 tablet.cc:1583] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush Phase 2: carrying over any updates which arrived during Phase 1 I0507 20:10:39.643370 546 tablet.cc:1585] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6249363615459508224 or (T in {6249363615459508224})}] W0507 20:10:39.652204 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 5 times. W0507 20:10:39.652320 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 5 times. W0507 20:10:39.658082 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 5 times. I0507 20:10:39.658789 546 tablet.cc:1633] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush successful on 10 rows (4506 bytes) I0507 20:10:39.659225 225 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet a0376a5e3b9f495aa00d366b35594a27 (attempt 2) I0507 20:10:39.659369 225 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a0376a5e3b9f495aa00d366b35594a27 with cas_config_opid_index -1 with a delay of 54 ms (attempt = 2) I0507 20:10:39.659920 546 maintenance_manager.cc:494] P b5e887ddd8a14cf3b2c24a0aa14be025: FlushMRSOp(7161aaf1a71d455883ef478337dfaf19) complete. Timing: real 0.195s user 0.032s sys 0.028s Metrics: {"cfile_init":1,"compiler_manager_pool.queue_time_us":38718,"data dirs.queue_time_us":11445,"data dirs.run_cpu_time_us":279,"data dirs.run_wall_time_us":1297,"lbm_read_time_us":68,"lbm_reads_lt_1ms":4,"lbm_write_time_us":1837,"lbm_writes_1-10_ms":1,"lbm_writes_lt_1ms":17,"thread_start_us":56381,"threads_started":3,"wal-append.queue_time_us":7463} I0507 20:10:39.664530 652 maintenance_manager.cc:304] P b5e887ddd8a14cf3b2c24a0aa14be025: Scheduling FlushMRSOp(d80f4fae4b9f4d238f7f626507290390): perf score=0.089966 I0507 20:10:39.665215 546 tablet.cc:1152] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering stage 1 (old memrowset already frozen for inserts) I0507 20:10:39.665329 546 compaction.cc:913] Selected 1 rowsets to compact: I0507 20:10:39.665390 546 compaction.cc:916] memrowset(current size on disk: ~0 bytes) I0507 20:10:39.665449 546 tablet.cc:1154] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Memstore in-memory size: 763 bytes I0507 20:10:39.665561 546 tablet.cc:1443] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6249363615674081280 or (T in {6249363615674081280})}] I0507 20:10:39.668014 546 multi_column_writer.cc:98] Opened CFile writers for 2 column(s) W0507 20:10:39.671681 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 6 times. I0507 20:10:39.672775 546 tablet.cc:1528] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I0507 20:10:39.673051 546 tablet.cc:1583] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush Phase 2: carrying over any updates which arrived during Phase 1 I0507 20:10:39.673110 546 tablet.cc:1585] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6249363615674081280 or (T in {6249363615674081280})}] I0507 20:10:39.678411 546 tablet.cc:1633] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush successful on 8 rows (4479 bytes) I0507 20:10:39.678684 546 maintenance_manager.cc:494] P b5e887ddd8a14cf3b2c24a0aa14be025: FlushMRSOp(d80f4fae4b9f4d238f7f626507290390) complete. Timing: real 0.014s user 0.012s sys 0.000s Metrics: {"cfile_init":1,"data dirs.queue_time_us":38,"data dirs.run_cpu_time_us":166,"data dirs.run_wall_time_us":2028,"lbm_read_time_us":62,"lbm_reads_lt_1ms":4,"lbm_write_time_us":477,"lbm_writes_lt_1ms":18,"thread_start_us":250,"threads_started":1,"wal-append.queue_time_us":415} I0507 20:10:39.679558 652 maintenance_manager.cc:304] P b5e887ddd8a14cf3b2c24a0aa14be025: Scheduling UndoDeltaBlockGCOp(7161aaf1a71d455883ef478337dfaf19): 274 bytes on disk I0507 20:10:39.680845 546 maintenance_manager.cc:494] P b5e887ddd8a14cf3b2c24a0aa14be025: UndoDeltaBlockGCOp(7161aaf1a71d455883ef478337dfaf19) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":80,"lbm_reads_lt_1ms":4} I0507 20:10:39.681294 652 maintenance_manager.cc:304] P b5e887ddd8a14cf3b2c24a0aa14be025: Scheduling UndoDeltaBlockGCOp(d80f4fae4b9f4d238f7f626507290390): 246 bytes on disk W0507 20:10:39.681337 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 6 times. W0507 20:10:39.681437 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 6 times. I0507 20:10:39.681771 546 maintenance_manager.cc:494] P b5e887ddd8a14cf3b2c24a0aa14be025: UndoDeltaBlockGCOp(d80f4fae4b9f4d238f7f626507290390) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":61,"lbm_reads_lt_1ms":4} I0507 20:10:39.682477 652 maintenance_manager.cc:304] P b5e887ddd8a14cf3b2c24a0aa14be025: Scheduling FlushMRSOp(a0376a5e3b9f495aa00d366b35594a27): perf score=0.089950 I0507 20:10:39.682924 546 tablet.cc:1152] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering stage 1 (old memrowset already frozen for inserts) I0507 20:10:39.682989 546 compaction.cc:913] Selected 1 rowsets to compact: I0507 20:10:39.683043 546 compaction.cc:916] memrowset(current size on disk: ~0 bytes) I0507 20:10:39.683082 546 tablet.cc:1154] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Memstore in-memory size: 763 bytes I0507 20:10:39.683152 546 tablet.cc:1443] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6249363610975170560 or (T in {6249363610975170560})}] I0507 20:10:39.684794 546 multi_column_writer.cc:98] Opened CFile writers for 2 column(s) I0507 20:10:39.688573 546 tablet.cc:1528] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I0507 20:10:39.688789 546 tablet.cc:1583] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush Phase 2: carrying over any updates which arrived during Phase 1 I0507 20:10:39.688870 546 tablet.cc:1585] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6249363610975170560 or (T in {6249363610975170560})}] I0507 20:10:39.691023 227 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet 7161aaf1a71d455883ef478337dfaf19 (attempt 2) I0507 20:10:39.691184 227 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7161aaf1a71d455883ef478337dfaf19 with cas_config_opid_index -1 with a delay of 48 ms (attempt = 2) W0507 20:10:39.692744 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 7 times. I0507 20:10:39.700389 546 tablet.cc:1633] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025: Flush successful on 6 rows (4492 bytes) I0507 20:10:39.700721 546 maintenance_manager.cc:494] P b5e887ddd8a14cf3b2c24a0aa14be025: FlushMRSOp(a0376a5e3b9f495aa00d366b35594a27) complete. Timing: real 0.018s user 0.008s sys 0.004s Metrics: {"cfile_init":1,"data dirs.queue_time_us":44,"data dirs.run_cpu_time_us":139,"data dirs.run_wall_time_us":1841,"lbm_read_time_us":66,"lbm_reads_lt_1ms":4,"lbm_write_time_us":395,"lbm_writes_lt_1ms":18,"thread_start_us":1449,"threads_started":1,"wal-append.queue_time_us":6931} I0507 20:10:39.701483 652 maintenance_manager.cc:304] P b5e887ddd8a14cf3b2c24a0aa14be025: Scheduling UndoDeltaBlockGCOp(a0376a5e3b9f495aa00d366b35594a27): 223 bytes on disk I0507 20:10:39.702059 546 maintenance_manager.cc:494] P b5e887ddd8a14cf3b2c24a0aa14be025: UndoDeltaBlockGCOp(a0376a5e3b9f495aa00d366b35594a27) complete. Timing: real 0.000s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4} W0507 20:10:39.705302 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 7 times. W0507 20:10:39.707520 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 7 times. I0507 20:10:39.714033 225 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet a0376a5e3b9f495aa00d366b35594a27 (attempt 3) I0507 20:10:39.714171 225 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a0376a5e3b9f495aa00d366b35594a27 with cas_config_opid_index -1 with a delay of 100 ms (attempt = 3) W0507 20:10:39.714318 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 8 times. W0507 20:10:39.727850 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 8 times. W0507 20:10:39.729682 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 8 times. W0507 20:10:39.738291 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 9 times. I0507 20:10:39.739722 227 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet 7161aaf1a71d455883ef478337dfaf19 (attempt 3) I0507 20:10:39.739864 227 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7161aaf1a71d455883ef478337dfaf19 with cas_config_opid_index -1 with a delay of 76 ms (attempt = 3) I0507 20:10:39.756232 211 external_mini_cluster.cc:978] Killing /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver with pid 277 W0507 20:10:39.757295 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 9 times. W0507 20:10:39.759642 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 9 times. W0507 20:10:39.765102 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 10 times. W0507 20:10:39.776624 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 10 times. W0507 20:10:39.787830 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 10 times. W0507 20:10:39.789438 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 11 times. W0507 20:10:39.799899 224 connection.cc:512] server connection from 127.0.52.193:59348 recv error: Network error: failed to read from TLS socket (remote: 127.0.52.193:59348): Connection reset by peer (error 104) W0507 20:10:39.800387 216 connection.cc:512] client connection to 127.0.52.193:46057 recv error: Network error: failed to read from TLS socket (remote: 127.0.52.193:46057): Connection reset by peer (error 104) W0507 20:10:39.800776 1420 negotiation.cc:313] Failed RPC negotiation. Trace: 0507 20:10:39.596637 (+ 0us) reactor.cc:577] Submitting negotiation task for client connection to 127.0.52.193:46057 0507 20:10:39.615295 (+ 18658us) negotiation.cc:98] Waiting for socket to connect 0507 20:10:39.615321 (+ 26us) client_negotiation.cc:160] Beginning negotiation 0507 20:10:39.615488 (+ 167us) client_negotiation.cc:237] Sending NEGOTIATE NegotiatePB request 0507 20:10:39.756406 (+140918us) client_negotiation.cc:254] Received NEGOTIATE NegotiatePB response 0507 20:10:39.756413 (+ 7us) client_negotiation.cc:348] Received NEGOTIATE response from server 0507 20:10:39.756464 (+ 51us) client_negotiation.cc:175] Negotiated authn=CERTIFICATE 0507 20:10:39.756648 (+ 184us) client_negotiation.cc:476] Received TLS_HANDSHAKE response from server 0507 20:10:39.756764 (+ 116us) client_negotiation.cc:464] Sending TLS_HANDSHAKE message to server 0507 20:10:39.756777 (+ 13us) client_negotiation.cc:237] Sending TLS_HANDSHAKE NegotiatePB request 0507 20:10:39.800172 (+ 43395us) negotiation.cc:304] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.0.52.193:46057: BlockingRecv error: recv error from 0.0.0.0:0: Transport endpoint is not connected (error 107) Metrics: {"client-negotiator.queue_time_us":16518,"thread_start_us":2817,"threads_started":1} W0507 20:10:39.801121 542 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Couldn't send request to peer d1bf0f72a88f4483a1906bc8246849dd for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.193:46057: BlockingRecv error: recv error from 0.0.0.0:0: Transport endpoint is not connected (error 107). Retrying in the next heartbeat period. Already tried 1 times. I0507 20:10:39.801313 211 external_mini_cluster.cc:1019] Attempting to check leaks for /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver pid 532 W0507 20:10:39.801208 542 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Couldn't send request to peer d1bf0f72a88f4483a1906bc8246849dd for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.193:46057: BlockingRecv error: recv error from 0.0.0.0:0: Transport endpoint is not connected (error 107). Retrying in the next heartbeat period. Already tried 1 times. W0507 20:10:39.801923 542 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Couldn't send request to peer d1bf0f72a88f4483a1906bc8246849dd for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.193:46057: BlockingRecv error: recv error from 0.0.0.0:0: Transport endpoint is not connected (error 107). Retrying in the next heartbeat period. Already tried 1 times. W0507 20:10:39.804787 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 11 times. I0507 20:10:39.809710 648 generic_service.cc:169] Checking for leaks (request via RPC) I0507 20:10:39.814754 225 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet a0376a5e3b9f495aa00d366b35594a27 (attempt 4) I0507 20:10:39.814915 225 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a0376a5e3b9f495aa00d366b35594a27 with cas_config_opid_index -1 with a delay of 137 ms (attempt = 4) I0507 20:10:39.816285 227 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet 7161aaf1a71d455883ef478337dfaf19 (attempt 4) I0507 20:10:39.816416 227 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7161aaf1a71d455883ef478337dfaf19 with cas_config_opid_index -1 with a delay of 155 ms (attempt = 4) I0507 20:10:39.952697 225 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet a0376a5e3b9f495aa00d366b35594a27 (attempt 5) I0507 20:10:39.952873 225 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a0376a5e3b9f495aa00d366b35594a27 with cas_config_opid_index -1 with a delay of 256 ms (attempt = 5) I0507 20:10:39.972180 227 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet 7161aaf1a71d455883ef478337dfaf19 (attempt 5) I0507 20:10:39.972337 227 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7161aaf1a71d455883ef478337dfaf19 with cas_config_opid_index -1 with a delay of 259 ms (attempt = 5) W0507 20:10:39.993713 542 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Couldn't send request to peer d1bf0f72a88f4483a1906bc8246849dd for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.193:46057: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 2 times. W0507 20:10:39.994488 541 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 12 times. W0507 20:10:39.994545 542 consensus_peers.cc:434] T d80f4fae4b9f4d238f7f626507290390 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Couldn't send request to peer d1bf0f72a88f4483a1906bc8246849dd for tablet d80f4fae4b9f4d238f7f626507290390. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.193:46057: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 2 times. W0507 20:10:39.994964 541 consensus_peers.cc:434] T a0376a5e3b9f495aa00d366b35594a27 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet a0376a5e3b9f495aa00d366b35594a27. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 11 times. W0507 20:10:39.995620 542 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer d1bf0f72a88f4483a1906bc8246849dd (127.0.52.193:46057): Couldn't send request to peer d1bf0f72a88f4483a1906bc8246849dd for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.193:46057: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 2 times. W0507 20:10:39.996217 541 consensus_peers.cc:434] T 7161aaf1a71d455883ef478337dfaf19 P b5e887ddd8a14cf3b2c24a0aa14be025 -> Peer 5014824b94b64fd49da2c1a645c078aa (127.0.52.194:43942): Couldn't send request to peer 5014824b94b64fd49da2c1a645c078aa for tablet 7161aaf1a71d455883ef478337dfaf19. Status: Network error: Client connection negotiation failed: client connection to 127.0.52.194:43942: connect: Connection refused (error 111). Retrying in the next heartbeat period. Already tried 12 times. I0507 20:10:39.997011 211 external_mini_cluster.cc:978] Killing /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-tserver with pid 532 W0507 20:10:40.025382 216 connection.cc:512] client connection to 127.0.52.195:38453 recv error: Network error: failed to read from TLS socket (remote: 127.0.52.195:38453): Connection reset by peer (error 104) W0507 20:10:40.025410 226 connection.cc:512] server connection from 127.0.52.195:50400 recv error: Network error: failed to read from TLS socket (remote: 0.0.0.0:0): Connection reset by peer (error 104) W0507 20:10:40.025694 226 connection.cc:185] Error closing socket: Network error: TlsSocket::Close: Broken pipe I0507 20:10:40.025715 211 external_mini_cluster.cc:1019] Attempting to check leaks for /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-master pid 217 I0507 20:10:40.031585 273 generic_service.cc:169] Checking for leaks (request via RPC) I0507 20:10:40.348381 227 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet 7161aaf1a71d455883ef478337dfaf19 (attempt 6) I0507 20:10:40.349040 225 catalog_manager.cc:3355] Sending ChangeConfig:ADD_PEER:NON_VOTER on tablet a0376a5e3b9f495aa00d366b35594a27 (attempt 6) I0507 20:10:40.349045 227 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet 7161aaf1a71d455883ef478337dfaf19 with cas_config_opid_index -1 with a delay of 559 ms (attempt = 6) I0507 20:10:40.349158 225 catalog_manager.cc:2821] Scheduling retry of ChangeConfig:ADD_PEER:NON_VOTER RPC for tablet a0376a5e3b9f495aa00d366b35594a27 with cas_config_opid_index -1 with a delay of 544 ms (attempt = 6) I0507 20:10:40.350190 211 external_mini_cluster.cc:978] Killing /tmp/dist-test-taskgGZ5OO/build/asan/bin/kudu-master with pid 217 W0507 20:10:40.403996 216 connection.cc:512] client connection to 127.0.52.254:52118 recv error: Network error: failed to read from TLS socket (remote: 127.0.52.254:52118): Connection reset by peer (error 104) I0507 20:10:40.406689 211 test_util.cc:123] ----------------------------------------------- I0507 20:10:40.406808 211 test_util.cc:124] Had fatal failures, leaving test files at /tmp/dist-test-taskgGZ5OO/test-tmp/client-negotiation-failover-itest.0.ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer.1525723514398967-211 [ FAILED ] ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer (325968 ms) [----------] 3 tests from ClientFailoverOnNegotiationTimeoutITest (343706 ms total) [ FAILED ] 1 test, listed below: [ FAILED ] ClientFailoverOnNegotiationTimeoutITest.Kudu1580ConnectToTServer 1 FAILED TEST