[==========] Running 3 tests from 1 test case. [----------] Global test environment set-up. [----------] 3 tests from DeleteTabletITest [ RUN ] DeleteTabletITest.TestDeleteFailedReplica WARNING: Logging before InitGoogleLogging() is written to STDERR I0513 22:33:14.628971 466 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.0.116.190:36371 I0513 22:33:14.636322 466 system_ntp.cc:202] NTP initialized. Skew: 500ppm Current error: 40684us I0513 22:33:14.636977 466 fs_manager.cc:263] Metadata directory not provided I0513 22:33:14.637061 466 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0513 22:33:14.637352 466 server_base.cc:433] Could not load existing FS layout: Not found: could not find a healthy instance file I0513 22:33:14.637446 466 server_base.cc:434] Attempting to create new FS layout instead I0513 22:33:14.641072 466 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/master-0-root/instance: uuid: "6281f2abcf5d4d77b8599dcf6658aa44" format_stamp: "Formatted at 2019-05-13 22:33:14 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:14.648964 466 fs_manager.cc:503] Time spent creating directory manager: real 0.007s user 0.007s sys 0.001s I0513 22:33:14.650084 466 env_posix.cc:1676] Not raising this process' open files per process limit of 1048576; it is already as high as it can go I0513 22:33:14.650792 466 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0513 22:33:14.654273 466 fs_manager.cc:419] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s I0513 22:33:14.654359 466 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/master-0-root uuid: "6281f2abcf5d4d77b8599dcf6658aa44" format_stamp: "Formatted at 2019-05-13 22:33:14 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:14.654525 466 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0513 22:33:14.707360 466 env_posix.cc:1676] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go I0513 22:33:14.725455 466 rpc_server.cc:205] RPC server started. Bound to: 127.0.116.190:36371 I0513 22:33:14.727054 466 webserver.cc:308] Webserver started at http://127.0.116.190:42609/ using document root and password file I0513 22:33:14.730959 525 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:14.743176 525 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44: Bootstrap starting. I0513 22:33:14.746264 525 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44: No blocks or log segments found. Creating new log. I0513 22:33:14.747229 525 log.cc:525] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44: Log is configured to *not* fsync() on all Append() calls I0513 22:33:14.749769 525 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44: No bootstrap required, opened a new log I0513 22:33:14.756515 525 raft_consensus.cc:340] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" member_type: VOTER } I0513 22:33:14.756806 525 raft_consensus.cc:366] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:14.756875 525 raft_consensus.cc:705] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6281f2abcf5d4d77b8599dcf6658aa44, State: Initialized, Role: FOLLOWER I0513 22:33:14.757496 525 consensus_queue.cc:229] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" member_type: VOTER } I0513 22:33:14.757690 525 raft_consensus.cc:380] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0513 22:33:14.757766 525 raft_consensus.cc:472] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0513 22:33:14.757884 525 raft_consensus.cc:2886] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:14.759626 525 raft_consensus.cc:494] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" member_type: VOTER } I0513 22:33:14.760155 525 leader_election.cc:310] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6281f2abcf5d4d77b8599dcf6658aa44; no voters: I0513 22:33:14.760898 525 leader_election.cc:296] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [CANDIDATE]: Term 1 election: Requested vote from peers I0513 22:33:14.761065 527 raft_consensus.cc:2640] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:14.762776 527 raft_consensus.cc:667] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 1 LEADER]: Becoming Leader. State: Replica: 6281f2abcf5d4d77b8599dcf6658aa44, State: Running, Role: LEADER I0513 22:33:14.763228 527 consensus_queue.cc:206] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" member_type: VOTER } I0513 22:33:14.765848 525 sys_catalog.cc:337] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" member_type: VOTER } } I0513 22:33:14.766140 525 sys_catalog.cc:340] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [sys.catalog]: This master's current role is: LEADER I0513 22:33:14.766089 529 sys_catalog.cc:337] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6281f2abcf5d4d77b8599dcf6658aa44. Latest consensus state: current_term: 1 leader_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" member_type: VOTER } } I0513 22:33:14.766377 529 sys_catalog.cc:340] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [sys.catalog]: This master's current role is: LEADER I0513 22:33:14.766258 528 sys_catalog.cc:337] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6281f2abcf5d4d77b8599dcf6658aa44" member_type: VOTER } } I0513 22:33:14.766631 528 sys_catalog.cc:340] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [sys.catalog]: This master's current role is: LEADER I0513 22:33:14.768808 525 sys_catalog.cc:424] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [sys.catalog]: configured and running, proceeding with master startup. I0513 22:33:14.768959 531 catalog_manager.cc:1055] Loading table and tablet metadata into memory... I0513 22:33:14.769330 466 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 I0513 22:33:14.774243 531 catalog_manager.cc:1066] Initializing Kudu internal certificate authority... I0513 22:33:14.796823 531 catalog_manager.cc:950] Generated new certificate authority record I0513 22:33:14.798172 531 catalog_manager.cc:1075] Loading token signing keys... I0513 22:33:14.808182 531 catalog_manager.cc:4292] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44: Generated new TSK 0 I0513 22:33:14.839800 466 system_ntp.cc:202] NTP initialized. Skew: 500ppm Current error: 40684us I0513 22:33:14.840046 466 fs_manager.cc:263] Metadata directory not provided I0513 22:33:14.840124 466 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0513 22:33:14.840261 466 server_base.cc:433] Could not load existing FS layout: Not found: could not find a healthy instance file I0513 22:33:14.840445 466 server_base.cc:434] Attempting to create new FS layout instead I0513 22:33:14.841369 466 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root/instance: uuid: "7c8a743061cb4dafa98b890a9d14a903" format_stamp: "Formatted at 2019-05-13 22:33:14 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:14.845257 466 fs_manager.cc:503] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s I0513 22:33:14.847640 466 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0513 22:33:14.849802 466 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0513 22:33:14.849866 466 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root uuid: "7c8a743061cb4dafa98b890a9d14a903" format_stamp: "Formatted at 2019-05-13 22:33:14 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:14.849982 466 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0513 22:33:14.885373 466 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0513 22:33:14.885530 466 ts_tablet_manager.cc:376] Registered 0 tablets I0513 22:33:14.924480 466 rpc_server.cc:205] RPC server started. Bound to: 127.0.116.129:41623 I0513 22:33:14.926210 466 webserver.cc:308] Webserver started at http://127.0.116.129:42871/ using document root and password file I0513 22:33:14.947778 600 heartbeater.cc:345] Connected to a master server at 127.0.116.190:36371 I0513 22:33:14.948082 600 heartbeater.cc:426] Registering TS with master... I0513 22:33:14.948729 600 heartbeater.cc:475] Master 127.0.116.190:36371 requested a full tablet report, sending... I0513 22:33:14.950986 492 ts_manager.cc:159] Registered new tserver with Master: 7c8a743061cb4dafa98b890a9d14a903 (127.0.116.129:41623) I0513 22:33:14.951414 466 internal_mini_cluster.cc:328] 1 TS(s) registered with all masters after 0.023158833s I0513 22:33:14.953348 492 master_service.cc:273] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:45720 I0513 22:33:14.979204 466 test_util.cc:212] Using random seed: -2138247351 I0513 22:33:14.993770 492 catalog_manager.cc:1409] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45726: name: "test-workload" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 1 split_rows_range_bounds { } partition_schema { range_schema { columns { name: "key" } } } I0513 22:33:15.025264 567 tablet_service.cc:1016] Processing CreateTablet for tablet 9479ddfbe2bd46158a9fd329f76a56e2 (table=test-workload [id=3a225cecabac48b9beb076b06406c385]), partition=RANGE (key) PARTITION UNBOUNDED I0513 22:33:15.025987 567 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 9479ddfbe2bd46158a9fd329f76a56e2. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.035637 618 tablet_bootstrap.cc:439] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Bootstrap starting. I0513 22:33:15.037804 618 tablet_bootstrap.cc:590] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: No blocks or log segments found. Creating new log. I0513 22:33:15.041558 618 tablet_bootstrap.cc:439] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: No bootstrap required, opened a new log I0513 22:33:15.041762 618 ts_tablet_manager.cc:1053] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s I0513 22:33:15.042675 618 raft_consensus.cc:340] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a743061cb4dafa98b890a9d14a903" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 41623 } } I0513 22:33:15.042915 618 raft_consensus.cc:366] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:15.042994 618 raft_consensus.cc:705] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c8a743061cb4dafa98b890a9d14a903, State: Initialized, Role: FOLLOWER I0513 22:33:15.043200 618 consensus_queue.cc:229] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a743061cb4dafa98b890a9d14a903" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 41623 } } I0513 22:33:15.043342 618 raft_consensus.cc:380] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0513 22:33:15.043396 618 raft_consensus.cc:472] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0513 22:33:15.043491 618 raft_consensus.cc:2886] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:15.045169 618 raft_consensus.cc:494] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a743061cb4dafa98b890a9d14a903" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 41623 } } I0513 22:33:15.045415 618 leader_election.cc:310] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7c8a743061cb4dafa98b890a9d14a903; no voters: I0513 22:33:15.045877 618 leader_election.cc:296] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [CANDIDATE]: Term 1 election: Requested vote from peers I0513 22:33:15.046099 620 raft_consensus.cc:2640] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:15.047960 600 heartbeater.cc:467] Master 127.0.116.190:36371 was elected leader, sending a full tablet report... I0513 22:33:15.048574 618 ts_tablet_manager.cc:1081] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Time spent starting tablet: real 0.007s user 0.000s sys 0.006s I0513 22:33:15.049180 620 raft_consensus.cc:667] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 1 LEADER]: Becoming Leader. State: Replica: 7c8a743061cb4dafa98b890a9d14a903, State: Running, Role: LEADER I0513 22:33:15.049466 620 consensus_queue.cc:206] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a743061cb4dafa98b890a9d14a903" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 41623 } } I0513 22:33:15.052325 492 catalog_manager.cc:3988] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 reported cstate change: term changed from 0 to 1, leader changed from to 7c8a743061cb4dafa98b890a9d14a903 (127.0.116.129). New cstate: current_term: 1 leader_uuid: "7c8a743061cb4dafa98b890a9d14a903" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7c8a743061cb4dafa98b890a9d14a903" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 41623 } health_report { overall_health: HEALTHY } } } I0513 22:33:15.150007 466 master.cc:269] Master@127.0.116.190:36371 shutting down... I0513 22:33:15.161728 466 raft_consensus.cc:2143] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 1 LEADER]: Raft consensus shutting down. I0513 22:33:15.162057 466 raft_consensus.cc:2170] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44 [term 1 FOLLOWER]: Raft consensus is shut down! I0513 22:33:15.162289 466 tablet_replica.cc:264] T 00000000000000000000000000000000 P 6281f2abcf5d4d77b8599dcf6658aa44: stopping tablet replica I0513 22:33:15.178511 466 master.cc:280] Master@127.0.116.190:36371 shutdown complete. I0513 22:33:15.179100 601 maintenance_manager.cc:306] P 7c8a743061cb4dafa98b890a9d14a903: Scheduling UndoDeltaBlockGCOp(9479ddfbe2bd46158a9fd329f76a56e2): 2736 bytes on disk I0513 22:33:15.180491 545 maintenance_manager.cc:501] P 7c8a743061cb4dafa98b890a9d14a903: UndoDeltaBlockGCOp(9479ddfbe2bd46158a9fd329f76a56e2) complete. Timing: real 0.001s user 0.000s sys 0.000s Metrics: {"cfile_init":1,"lbm_read_time_us":83,"lbm_reads_lt_1ms":4} I0513 22:33:15.184267 466 tablet_server.cc:165] TabletServer shutting down... I0513 22:33:15.196161 466 ts_tablet_manager.cc:1126] Shutting down tablet manager... I0513 22:33:15.196525 466 tablet_replica.cc:264] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: stopping tablet replica I0513 22:33:15.196707 466 raft_consensus.cc:2143] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 1 LEADER]: Raft consensus shutting down. I0513 22:33:15.196892 466 raft_consensus.cc:2170] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 1 FOLLOWER]: Raft consensus is shut down! I0513 22:33:15.213868 466 tablet_server.cc:179] TabletServer shutdown complete. I0513 22:33:15.213976 466 tablet_server.cc:165] TabletServer shutting down... I0513 22:33:15.214229 466 tablet_server.cc:179] TabletServer shutdown complete. I0513 22:33:15.222254 466 system_ntp.cc:202] NTP initialized. Skew: 500ppm Current error: 41184us I0513 22:33:15.222509 466 fs_manager.cc:263] Metadata directory not provided I0513 22:33:15.222599 466 fs_manager.cc:266] Using existing metadata directory in first data directory I0513 22:33:15.224894 466 fs_manager.cc:399] Time spent opening directory manager: real 0.002s user 0.000s sys 0.001s I0513 22:33:15.225908 466 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0513 22:33:15.232127 466 fs_manager.cc:419] Time spent opening block manager: real 0.005s user 0.000s sys 0.000s I0513 22:33:15.232244 466 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root uuid: "7c8a743061cb4dafa98b890a9d14a903" format_stamp: "Formatted at 2019-05-13 22:33:14 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.232378 466 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root/data Total live blocks: 5 Total live bytes: 15276 Total live bytes (after alignment): 28672 Total number of LBM containers: 6 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0513 22:33:15.332693 466 ts_tablet_manager.cc:344] Loading tablet metadata (0/1 complete) I0513 22:33:15.336413 466 ts_tablet_manager.cc:356] Loaded tablet metadata (1 total tablets, 1 live tablets) I0513 22:33:15.336669 466 ts_tablet_manager.cc:362] Registering tablets (0/1 complete) I0513 22:33:15.339027 466 ts_tablet_manager.cc:376] Registered 1 tablets I0513 22:33:15.339263 643 tablet_bootstrap.cc:439] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Bootstrap starting. W0513 22:33:15.348352 643 log.cc:1166] Error closing log: Illegal state: Log not open. State: 0 E0513 22:33:15.348980 643 ts_tablet_manager.cc:1074] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Tablet failed to bootstrap: IO error: Failed log replay. Reason: Failed to open new log: Insufficient disk space to allocate 8388608 bytes under path /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root/wals/9479ddfbe2bd46158a9fd329f76a56e2/.kudutmp.newsegmentWkJn2x (46931755008 bytes available vs 9223372036854775807 bytes reserved) (error 28) I0513 22:33:15.349078 643 ts_tablet_manager.cc:1053] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s I0513 22:33:15.349200 643 tablet_replica.cc:264] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: stopping tablet replica I0513 22:33:15.349261 643 raft_consensus.cc:2143] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 1 FOLLOWER]: Raft consensus shutting down. I0513 22:33:15.349359 643 raft_consensus.cc:2170] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903 [term 1 FOLLOWER]: Raft consensus is shut down! I0513 22:33:15.365224 466 rpc_server.cc:205] RPC server started. Bound to: 127.0.116.129:41623 I0513 22:33:15.366497 466 webserver.cc:308] Webserver started at http://127.0.116.129:42871/ using document root and password file W0513 22:33:15.369221 699 heartbeater.cc:587] Failed to heartbeat to 127.0.116.190:36371 (0 consecutive failures): Network error: Failed to ping master at 127.0.116.190:36371: Client connection negotiation failed: client connection to 127.0.116.190:36371: connect: Connection refused (error 111) W0513 22:33:15.370894 699 heartbeater.cc:380] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts. I0513 22:33:15.376204 666 tablet_service.cc:1057] Processing DeleteTablet for tablet 9479ddfbe2bd46158a9fd329f76a56e2 with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:49654 I0513 22:33:15.380813 703 ts_tablet_manager.cc:1377] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Deleting tablet data with delete state TABLET_DATA_DELETED I0513 22:33:15.385833 703 ts_tablet_manager.cc:1390] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown I0513 22:33:15.385963 703 log.cc:990] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestDeleteFailedReplica.1557786794576917-466/minicluster-data/ts-0-root/wals/9479ddfbe2bd46158a9fd329f76a56e2 I0513 22:33:15.387221 703 ts_tablet_manager.cc:1411] T 9479ddfbe2bd46158a9fd329f76a56e2 P 7c8a743061cb4dafa98b890a9d14a903: Deleting consensus metadata I0513 22:33:15.389936 466 tablet_server.cc:165] TabletServer shutting down... I0513 22:33:15.402300 466 ts_tablet_manager.cc:1126] Shutting down tablet manager... I0513 22:33:15.415938 466 tablet_server.cc:179] TabletServer shutdown complete. I0513 22:33:15.416007 466 tablet_server.cc:165] TabletServer shutting down... I0513 22:33:15.416185 466 tablet_server.cc:179] TabletServer shutdown complete. [ OK ] DeleteTabletITest.TestDeleteFailedReplica (799 ms) [ RUN ] DeleteTabletITest.TestLeaderElectionDuringDeleteTablet I0513 22:33:15.426645 466 internal_mini_cluster.cc:146] Creating distributed mini masters. Addrs: 127.0.116.190:38039 I0513 22:33:15.430476 466 system_ntp.cc:202] NTP initialized. Skew: 500ppm Current error: 41184us I0513 22:33:15.430747 466 fs_manager.cc:263] Metadata directory not provided I0513 22:33:15.430847 466 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0513 22:33:15.431006 466 server_base.cc:433] Could not load existing FS layout: Not found: could not find a healthy instance file I0513 22:33:15.431121 466 server_base.cc:434] Attempting to create new FS layout instead I0513 22:33:15.431887 466 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/master-0-root/instance: uuid: "59ca47066a8040f1aa351ed908a97b21" format_stamp: "Formatted at 2019-05-13 22:33:15 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.436004 466 fs_manager.cc:503] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s I0513 22:33:15.437247 466 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0513 22:33:15.439541 466 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0513 22:33:15.439617 466 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/master-0-root uuid: "59ca47066a8040f1aa351ed908a97b21" format_stamp: "Formatted at 2019-05-13 22:33:15 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.439757 466 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/master-0-root metadata directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/master-0-root 1 data directories: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/master-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0513 22:33:15.499693 466 rpc_server.cc:205] RPC server started. Bound to: 127.0.116.190:38039 I0513 22:33:15.503144 466 webserver.cc:308] Webserver started at http://127.0.116.190:35835/ using document root and password file I0513 22:33:15.505285 758 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.509074 758 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21: Bootstrap starting. I0513 22:33:15.510579 758 tablet_bootstrap.cc:590] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21: No blocks or log segments found. Creating new log. I0513 22:33:15.513572 758 tablet_bootstrap.cc:439] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21: No bootstrap required, opened a new log I0513 22:33:15.514322 758 raft_consensus.cc:340] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ca47066a8040f1aa351ed908a97b21" member_type: VOTER } I0513 22:33:15.514458 758 raft_consensus.cc:366] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:15.514583 758 raft_consensus.cc:705] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59ca47066a8040f1aa351ed908a97b21, State: Initialized, Role: FOLLOWER I0513 22:33:15.514840 758 consensus_queue.cc:229] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ca47066a8040f1aa351ed908a97b21" member_type: VOTER } I0513 22:33:15.515054 758 raft_consensus.cc:380] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately I0513 22:33:15.515107 758 raft_consensus.cc:472] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration) I0513 22:33:15.515208 758 raft_consensus.cc:2886] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:15.517379 758 raft_consensus.cc:494] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ca47066a8040f1aa351ed908a97b21" member_type: VOTER } I0513 22:33:15.517603 758 leader_election.cc:310] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59ca47066a8040f1aa351ed908a97b21; no voters: I0513 22:33:15.518169 760 raft_consensus.cc:2640] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:15.518461 758 leader_election.cc:296] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [CANDIDATE]: Term 1 election: Requested vote from peers I0513 22:33:15.520016 760 raft_consensus.cc:667] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [term 1 LEADER]: Becoming Leader. State: Replica: 59ca47066a8040f1aa351ed908a97b21, State: Running, Role: LEADER I0513 22:33:15.520282 760 consensus_queue.cc:206] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ca47066a8040f1aa351ed908a97b21" member_type: VOTER } I0513 22:33:15.521436 758 sys_catalog.cc:337] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [sys.catalog]: SysCatalogTable state changed. Reason: Started TabletReplica. Latest consensus state: current_term: 1 leader_uuid: "59ca47066a8040f1aa351ed908a97b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ca47066a8040f1aa351ed908a97b21" member_type: VOTER } } I0513 22:33:15.521550 761 sys_catalog.cc:337] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59ca47066a8040f1aa351ed908a97b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ca47066a8040f1aa351ed908a97b21" member_type: VOTER } } I0513 22:33:15.522264 761 sys_catalog.cc:340] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [sys.catalog]: This master's current role is: LEADER I0513 22:33:15.521972 762 sys_catalog.cc:337] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59ca47066a8040f1aa351ed908a97b21. Latest consensus state: current_term: 1 leader_uuid: "59ca47066a8040f1aa351ed908a97b21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59ca47066a8040f1aa351ed908a97b21" member_type: VOTER } } I0513 22:33:15.522348 758 sys_catalog.cc:340] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [sys.catalog]: This master's current role is: LEADER I0513 22:33:15.522737 762 sys_catalog.cc:340] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [sys.catalog]: This master's current role is: LEADER I0513 22:33:15.523320 758 sys_catalog.cc:424] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21 [sys.catalog]: configured and running, proceeding with master startup. I0513 22:33:15.523910 466 internal_mini_cluster.cc:166] Waiting to initialize catalog manager on master 0 W0513 22:33:15.525542 766 catalog_manager.cc:1113] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21: acquiring CA information for follower catalog manager: Not found: root CA entry not found W0513 22:33:15.525619 766 catalog_manager.cc:596] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry I0513 22:33:15.525707 765 catalog_manager.cc:1055] Loading table and tablet metadata into memory... I0513 22:33:15.527662 765 catalog_manager.cc:1066] Initializing Kudu internal certificate authority... I0513 22:33:15.555127 765 catalog_manager.cc:950] Generated new certificate authority record I0513 22:33:15.557406 765 catalog_manager.cc:1075] Loading token signing keys... I0513 22:33:15.570119 765 catalog_manager.cc:4292] T 00000000000000000000000000000000 P 59ca47066a8040f1aa351ed908a97b21: Generated new TSK 0 I0513 22:33:15.591907 466 system_ntp.cc:202] NTP initialized. Skew: 500ppm Current error: 41184us I0513 22:33:15.592265 466 fs_manager.cc:263] Metadata directory not provided I0513 22:33:15.592432 466 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0513 22:33:15.592623 466 server_base.cc:433] Could not load existing FS layout: Not found: could not find a healthy instance file I0513 22:33:15.592705 466 server_base.cc:434] Attempting to create new FS layout instead I0513 22:33:15.593309 466 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/instance: uuid: "6fe2349260fd4c51a94a6bc60fa6043b" format_stamp: "Formatted at 2019-05-13 22:33:15 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.600473 466 fs_manager.cc:503] Time spent creating directory manager: real 0.007s user 0.000s sys 0.006s I0513 22:33:15.601707 466 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0513 22:33:15.603891 466 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s I0513 22:33:15.603965 466 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root uuid: "6fe2349260fd4c51a94a6bc60fa6043b" format_stamp: "Formatted at 2019-05-13 22:33:15 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.604074 466 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root metadata directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root 1 data directories: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0513 22:33:15.642338 466 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0513 22:33:15.642412 466 ts_tablet_manager.cc:376] Registered 0 tablets I0513 22:33:15.657038 466 rpc_server.cc:205] RPC server started. Bound to: 127.0.116.129:34317 I0513 22:33:15.657882 466 webserver.cc:308] Webserver started at http://127.0.116.129:34421/ using document root and password file I0513 22:33:15.661957 466 system_ntp.cc:202] NTP initialized. Skew: 500ppm Current error: 41184us I0513 22:33:15.662149 466 fs_manager.cc:263] Metadata directory not provided I0513 22:33:15.662199 466 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0513 22:33:15.662312 466 server_base.cc:433] Could not load existing FS layout: Not found: could not find a healthy instance file I0513 22:33:15.662348 466 server_base.cc:434] Attempting to create new FS layout instead I0513 22:33:15.662900 466 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-1-root/instance: uuid: "75a5cb20b56443639cfe49c47274d76b" format_stamp: "Formatted at 2019-05-13 22:33:15 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.666975 466 fs_manager.cc:503] Time spent creating directory manager: real 0.004s user 0.000s sys 0.002s I0513 22:33:15.668393 466 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0513 22:33:15.670483 466 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0513 22:33:15.670650 466 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-1-root uuid: "75a5cb20b56443639cfe49c47274d76b" format_stamp: "Formatted at 2019-05-13 22:33:15 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.670783 466 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-1-root metadata directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-1-root 1 data directories: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-1-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0513 22:33:15.674124 832 heartbeater.cc:345] Connected to a master server at 127.0.116.190:38039 I0513 22:33:15.674228 832 heartbeater.cc:426] Registering TS with master... I0513 22:33:15.674567 832 heartbeater.cc:475] Master 127.0.116.190:38039 requested a full tablet report, sending... I0513 22:33:15.675492 725 ts_manager.cc:159] Registered new tserver with Master: 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317) I0513 22:33:15.676618 725 master_service.cc:273] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58336 I0513 22:33:15.706238 466 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0513 22:33:15.706308 466 ts_tablet_manager.cc:376] Registered 0 tablets I0513 22:33:15.724730 466 rpc_server.cc:205] RPC server started. Bound to: 127.0.116.130:35393 I0513 22:33:15.725594 466 webserver.cc:308] Webserver started at http://127.0.116.130:43767/ using document root and password file I0513 22:33:15.732144 466 system_ntp.cc:202] NTP initialized. Skew: 500ppm Current error: 41184us I0513 22:33:15.732509 466 fs_manager.cc:263] Metadata directory not provided I0513 22:33:15.732616 466 fs_manager.cc:269] Using write-ahead log directory (fs_wal_dir) as metadata directory I0513 22:33:15.732738 466 server_base.cc:433] Could not load existing FS layout: Not found: could not find a healthy instance file I0513 22:33:15.732792 466 server_base.cc:434] Attempting to create new FS layout instead I0513 22:33:15.733496 466 fs_manager.cc:602] Generated new instance metadata in path /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-2-root/instance: uuid: "fb91e39117a5489e85c5a494ee14bcc3" format_stamp: "Formatted at 2019-05-13 22:33:15 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.735514 900 heartbeater.cc:345] Connected to a master server at 127.0.116.190:38039 I0513 22:33:15.735664 900 heartbeater.cc:426] Registering TS with master... I0513 22:33:15.736002 900 heartbeater.cc:475] Master 127.0.116.190:38039 requested a full tablet report, sending... I0513 22:33:15.737051 725 ts_manager.cc:159] Registered new tserver with Master: 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) I0513 22:33:15.738340 725 master_service.cc:273] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58338 I0513 22:33:15.739626 466 fs_manager.cc:503] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s I0513 22:33:15.740752 466 file_cache.cc:465] Constructed file cache lbm with capacity 419430 I0513 22:33:15.743248 466 fs_manager.cc:419] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s I0513 22:33:15.743316 466 fs_manager.cc:436] Opened local filesystem: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-2-root uuid: "fb91e39117a5489e85c5a494ee14bcc3" format_stamp: "Formatted at 2019-05-13 22:33:15 on dist-test-slave-dist-test-slave-bzsb" I0513 22:33:15.743453 466 fs_report.cc:352] FS layout report -------------------- wal directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-2-root metadata directory: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-2-root 1 data directories: /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-2-root/data Total live blocks: 0 Total live bytes: 0 Total live bytes (after alignment): 0 Total number of LBM containers: 0 (0 full) Did not check for missing blocks Did not check for orphaned blocks Total full LBM containers with extra space: 0 (0 repaired) Total full LBM container extra space in bytes: 0 (0 repaired) Total incomplete LBM containers: 0 (0 repaired) Total LBM partial records: 0 (0 repaired) I0513 22:33:15.758143 466 ts_tablet_manager.cc:356] Loaded tablet metadata (0 total tablets, 0 live tablets) I0513 22:33:15.758229 466 ts_tablet_manager.cc:376] Registered 0 tablets I0513 22:33:15.775528 466 rpc_server.cc:205] RPC server started. Bound to: 127.0.116.131:36837 I0513 22:33:15.776402 466 webserver.cc:308] Webserver started at http://127.0.116.131:39771/ using document root and password file I0513 22:33:15.783917 967 heartbeater.cc:345] Connected to a master server at 127.0.116.190:38039 I0513 22:33:15.784052 967 heartbeater.cc:426] Registering TS with master... I0513 22:33:15.784337 967 heartbeater.cc:475] Master 127.0.116.190:38039 requested a full tablet report, sending... I0513 22:33:15.785293 725 ts_manager.cc:159] Registered new tserver with Master: fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:15.786252 466 internal_mini_cluster.cc:328] 3 TS(s) registered with all masters after 0.009007127s I0513 22:33:15.786315 725 master_service.cc:273] Signed X509 certificate for tserver {username='slave'} at 127.0.0.1:58340 I0513 22:33:15.805474 466 test_util.cc:212] Using random seed: -2137421082 I0513 22:33:15.816946 725 catalog_manager.cc:1409] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58346: name: "test-workload" schema { columns { name: "key" type: INT32 is_key: true is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "int_val" type: INT32 is_key: false is_nullable: false encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } columns { name: "string_val" type: STRING is_key: false is_nullable: true encoding: AUTO_ENCODING compression: DEFAULT_COMPRESSION cfile_block_size: 0 } } num_replicas: 3 split_rows_range_bounds { rows: "\004\001\000\314\314\314\014\004\001\000\230\231\231\031\004\001\000dff&\004\001\0000333\004\001\000\374\377\377?\004\001\000\310\314\314L\004\001\000\224\231\231Y\004\001\000`fff\004\001\000,33s" } partition_schema { range_schema { columns { name: "key" } } } W0513 22:33:15.818118 725 catalog_manager.cc:1574] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-workload in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers. I0513 22:33:15.868602 866 tablet_service.cc:1016] Processing CreateTablet for tablet 9ce2cd8e3814425b86d4ef0b5309b05c (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728 I0513 22:33:15.868602 867 tablet_service.cc:1016] Processing CreateTablet for tablet c1cbbc2b2f22460aa0617e7a5c823a57 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION VALUES < 214748364 I0513 22:33:15.871299 866 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 9ce2cd8e3814425b86d4ef0b5309b05c. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.871958 865 tablet_service.cc:1016] Processing CreateTablet for tablet 8002a0fb94514f77a8601f629424436e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092 I0513 22:33:15.872654 863 tablet_service.cc:1016] Processing CreateTablet for tablet a40718aa091247438d353d8c7739041e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820 I0513 22:33:15.874440 867 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet c1cbbc2b2f22460aa0617e7a5c823a57. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.874533 861 tablet_service.cc:1016] Processing CreateTablet for tablet 7e57e922f3a84a2e99667b5eacf7c18e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548 I0513 22:33:15.875488 862 tablet_service.cc:1016] Processing CreateTablet for tablet e1484634c50f45f28345fdc882a61cf5 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184 I0513 22:33:15.875727 858 tablet_service.cc:1016] Processing CreateTablet for tablet 9fa827071e4246508d2690d2c1fbef4f (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION VALUES >= 1932735276 I0513 22:33:15.875061 864 tablet_service.cc:1016] Processing CreateTablet for tablet 3c141683c43f4fb78c4489e7904b28e8 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456 I0513 22:33:15.876953 865 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 8002a0fb94514f77a8601f629424436e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.876910 859 tablet_service.cc:1016] Processing CreateTablet for tablet 01c2af1f12034771ba026a1fac9203df (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276 I0513 22:33:15.876812 860 tablet_service.cc:1016] Processing CreateTablet for tablet 73f1725c5aa04760a9d5e43d09f66e9e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912 I0513 22:33:15.878696 863 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet a40718aa091247438d353d8c7739041e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.880508 932 tablet_service.cc:1016] Processing CreateTablet for tablet 8002a0fb94514f77a8601f629424436e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092 I0513 22:33:15.882467 858 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 9fa827071e4246508d2690d2c1fbef4f. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.883226 859 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 01c2af1f12034771ba026a1fac9203df. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.882673 794 tablet_service.cc:1016] Processing CreateTablet for tablet e1484634c50f45f28345fdc882a61cf5 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184 I0513 22:33:15.883545 797 tablet_service.cc:1016] Processing CreateTablet for tablet 8002a0fb94514f77a8601f629424436e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 429496728 <= VALUES < 644245092 I0513 22:33:15.884101 928 tablet_service.cc:1016] Processing CreateTablet for tablet 7e57e922f3a84a2e99667b5eacf7c18e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548 I0513 22:33:15.884129 931 tablet_service.cc:1016] Processing CreateTablet for tablet 3c141683c43f4fb78c4489e7904b28e8 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456 I0513 22:33:15.885607 864 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 3c141683c43f4fb78c4489e7904b28e8. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.884682 798 tablet_service.cc:1016] Processing CreateTablet for tablet 9ce2cd8e3814425b86d4ef0b5309b05c (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728 I0513 22:33:15.884919 929 tablet_service.cc:1016] Processing CreateTablet for tablet e1484634c50f45f28345fdc882a61cf5 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1073741820 <= VALUES < 1288490184 I0513 22:33:15.885460 933 tablet_service.cc:1016] Processing CreateTablet for tablet 9ce2cd8e3814425b86d4ef0b5309b05c (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 214748364 <= VALUES < 429496728 I0513 22:33:15.887454 794 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet e1484634c50f45f28345fdc882a61cf5. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.888211 932 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 8002a0fb94514f77a8601f629424436e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.887506 795 tablet_service.cc:1016] Processing CreateTablet for tablet a40718aa091247438d353d8c7739041e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820 I0513 22:33:15.889420 862 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet e1484634c50f45f28345fdc882a61cf5. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.888870 793 tablet_service.cc:1016] Processing CreateTablet for tablet 7e57e922f3a84a2e99667b5eacf7c18e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1288490184 <= VALUES < 1503238548 I0513 22:33:15.887506 796 tablet_service.cc:1016] Processing CreateTablet for tablet 3c141683c43f4fb78c4489e7904b28e8 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 644245092 <= VALUES < 858993456 I0513 22:33:15.890446 928 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 7e57e922f3a84a2e99667b5eacf7c18e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.889432 927 tablet_service.cc:1016] Processing CreateTablet for tablet 73f1725c5aa04760a9d5e43d09f66e9e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912 I0513 22:33:15.890043 934 tablet_service.cc:1016] Processing CreateTablet for tablet c1cbbc2b2f22460aa0617e7a5c823a57 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION VALUES < 214748364 I0513 22:33:15.891687 797 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 8002a0fb94514f77a8601f629424436e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.892166 860 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 73f1725c5aa04760a9d5e43d09f66e9e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.892792 798 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 9ce2cd8e3814425b86d4ef0b5309b05c. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.891912 925 tablet_service.cc:1016] Processing CreateTablet for tablet 9fa827071e4246508d2690d2c1fbef4f (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION VALUES >= 1932735276 I0513 22:33:15.893803 929 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet e1484634c50f45f28345fdc882a61cf5. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.894479 861 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 7e57e922f3a84a2e99667b5eacf7c18e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.894663 791 tablet_service.cc:1016] Processing CreateTablet for tablet 01c2af1f12034771ba026a1fac9203df (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276 I0513 22:33:15.893127 926 tablet_service.cc:1016] Processing CreateTablet for tablet 01c2af1f12034771ba026a1fac9203df (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1717986912 <= VALUES < 1932735276 I0513 22:33:15.893719 799 tablet_service.cc:1016] Processing CreateTablet for tablet c1cbbc2b2f22460aa0617e7a5c823a57 (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION VALUES < 214748364 I0513 22:33:15.892876 792 tablet_service.cc:1016] Processing CreateTablet for tablet 73f1725c5aa04760a9d5e43d09f66e9e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 1503238548 <= VALUES < 1717986912 I0513 22:33:15.896159 933 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 9ce2cd8e3814425b86d4ef0b5309b05c. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.896914 793 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 7e57e922f3a84a2e99667b5eacf7c18e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.895956 790 tablet_service.cc:1016] Processing CreateTablet for tablet 9fa827071e4246508d2690d2c1fbef4f (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION VALUES >= 1932735276 I0513 22:33:15.898705 927 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 73f1725c5aa04760a9d5e43d09f66e9e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.899233 795 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet a40718aa091247438d353d8c7739041e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.898507 930 tablet_service.cc:1016] Processing CreateTablet for tablet a40718aa091247438d353d8c7739041e (table=test-workload [id=168ff535840b4f83bfba5ca5121fb568]), partition=RANGE (key) PARTITION 858993456 <= VALUES < 1073741820 I0513 22:33:15.903159 925 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 9fa827071e4246508d2690d2c1fbef4f. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.904953 796 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 3c141683c43f4fb78c4489e7904b28e8. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.905494 799 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet c1cbbc2b2f22460aa0617e7a5c823a57. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.906183 790 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 9fa827071e4246508d2690d2c1fbef4f. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.907194 791 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 01c2af1f12034771ba026a1fac9203df. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.908326 926 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 01c2af1f12034771ba026a1fac9203df. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.908962 930 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet a40718aa091247438d353d8c7739041e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.909526 934 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet c1cbbc2b2f22460aa0617e7a5c823a57. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.909835 792 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 73f1725c5aa04760a9d5e43d09f66e9e. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.913362 931 data_dirs.cc:938] Could only allocate 1 dirs of requested 3 for tablet 3c141683c43f4fb78c4489e7904b28e8. 1 dirs total, 0 dirs full, 0 dirs failed I0513 22:33:15.995400 989 tablet_bootstrap.cc:439] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.010082 988 tablet_bootstrap.cc:439] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.013193 990 tablet_bootstrap.cc:439] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.016765 989 tablet_bootstrap.cc:590] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.019582 989 tablet_bootstrap.cc:439] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.019735 989 ts_tablet_manager.cc:1053] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.030s user 0.000s sys 0.005s I0513 22:33:16.020154 988 tablet_bootstrap.cc:590] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.020362 990 tablet_bootstrap.cc:590] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.020705 989 raft_consensus.cc:340] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.020948 989 raft_consensus.cc:366] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.021030 989 raft_consensus.cc:705] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.021200 989 consensus_queue.cc:229] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.021878 967 heartbeater.cc:467] Master 127.0.116.190:38039 was elected leader, sending a full tablet report... I0513 22:33:16.022392 989 ts_tablet_manager.cc:1081] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s I0513 22:33:16.022668 989 tablet_bootstrap.cc:439] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.024207 988 tablet_bootstrap.cc:439] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.025151 988 ts_tablet_manager.cc:1053] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.019s user 0.000s sys 0.005s I0513 22:33:16.025360 990 tablet_bootstrap.cc:439] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.025524 990 ts_tablet_manager.cc:1053] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.000s I0513 22:33:16.025843 989 tablet_bootstrap.cc:590] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.025776 988 raft_consensus.cc:340] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.025961 988 raft_consensus.cc:366] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.026015 988 raft_consensus.cc:705] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.026196 988 consensus_queue.cc:229] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.027168 900 heartbeater.cc:467] Master 127.0.116.190:38039 was elected leader, sending a full tablet report... I0513 22:33:16.026715 990 raft_consensus.cc:340] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } W0513 22:33:16.028033 968 tablet.cc:1791] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3: Can't schedule compaction. Clean time has not been advanced past its initial value. I0513 22:33:16.028378 990 raft_consensus.cc:366] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.028478 988 ts_tablet_manager.cc:1081] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s I0513 22:33:16.028811 990 raft_consensus.cc:705] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.029115 988 tablet_bootstrap.cc:439] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.029088 990 consensus_queue.cc:229] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.029580 989 tablet_bootstrap.cc:439] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.029709 989 ts_tablet_manager.cc:1053] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s I0513 22:33:16.030272 989 raft_consensus.cc:340] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.030462 989 raft_consensus.cc:366] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.030509 989 raft_consensus.cc:705] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.030923 989 consensus_queue.cc:229] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.031163 988 tablet_bootstrap.cc:590] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.032263 989 ts_tablet_manager.cc:1081] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s I0513 22:33:16.032476 989 tablet_bootstrap.cc:439] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.034042 832 heartbeater.cc:467] Master 127.0.116.190:38039 was elected leader, sending a full tablet report... I0513 22:33:16.034472 990 ts_tablet_manager.cc:1081] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.008s user 0.002s sys 0.004s I0513 22:33:16.035071 989 tablet_bootstrap.cc:590] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.035387 990 tablet_bootstrap.cc:439] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.036325 988 tablet_bootstrap.cc:439] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.036482 988 ts_tablet_manager.cc:1053] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.005s I0513 22:33:16.037119 988 raft_consensus.cc:340] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.037324 988 raft_consensus.cc:366] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.037361 988 raft_consensus.cc:705] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.037736 988 consensus_queue.cc:229] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.037981 990 tablet_bootstrap.cc:590] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.038815 988 ts_tablet_manager.cc:1081] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s I0513 22:33:16.039021 988 tablet_bootstrap.cc:439] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.040164 988 tablet_bootstrap.cc:590] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.041900 990 tablet_bootstrap.cc:439] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.042076 990 ts_tablet_manager.cc:1053] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.002s I0513 22:33:16.042295 989 tablet_bootstrap.cc:439] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.042374 988 tablet_bootstrap.cc:439] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.042445 989 ts_tablet_manager.cc:1053] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s I0513 22:33:16.042492 988 ts_tablet_manager.cc:1053] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s I0513 22:33:16.042838 990 raft_consensus.cc:340] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.044039 990 raft_consensus.cc:366] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.044018 988 raft_consensus.cc:340] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.044139 990 raft_consensus.cc:705] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.044152 988 raft_consensus.cc:366] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.044085 989 raft_consensus.cc:340] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.044226 988 raft_consensus.cc:705] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.044270 989 raft_consensus.cc:366] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.044365 989 raft_consensus.cc:705] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.044402 990 consensus_queue.cc:229] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.044443 988 consensus_queue.cc:229] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.044569 989 consensus_queue.cc:229] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.044919 990 ts_tablet_manager.cc:1081] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s I0513 22:33:16.045106 990 tablet_bootstrap.cc:439] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.045783 989 ts_tablet_manager.cc:1081] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s I0513 22:33:16.045971 989 tablet_bootstrap.cc:439] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.046559 990 tablet_bootstrap.cc:590] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.047083 988 ts_tablet_manager.cc:1081] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s I0513 22:33:16.047253 989 tablet_bootstrap.cc:590] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.047279 988 tablet_bootstrap.cc:439] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.048728 988 tablet_bootstrap.cc:590] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.049850 990 tablet_bootstrap.cc:439] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.050180 989 tablet_bootstrap.cc:439] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.050921 990 ts_tablet_manager.cc:1053] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s I0513 22:33:16.051087 989 ts_tablet_manager.cc:1053] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.001s I0513 22:33:16.051534 990 raft_consensus.cc:340] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.051630 988 tablet_bootstrap.cc:439] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.051713 990 raft_consensus.cc:366] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.051648 989 raft_consensus.cc:340] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.051818 990 raft_consensus.cc:705] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.051836 988 ts_tablet_manager.cc:1053] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s I0513 22:33:16.051879 989 raft_consensus.cc:366] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.051934 989 raft_consensus.cc:705] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.052021 990 consensus_queue.cc:229] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.052150 989 consensus_queue.cc:229] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.052526 988 raft_consensus.cc:340] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.052678 988 raft_consensus.cc:366] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.052717 988 raft_consensus.cc:705] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.052940 988 consensus_queue.cc:229] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.053217 989 ts_tablet_manager.cc:1081] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s I0513 22:33:16.053473 989 tablet_bootstrap.cc:439] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.054823 990 ts_tablet_manager.cc:1081] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s I0513 22:33:16.054996 989 tablet_bootstrap.cc:590] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.055029 990 tablet_bootstrap.cc:439] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.055618 988 ts_tablet_manager.cc:1081] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s I0513 22:33:16.055860 988 tablet_bootstrap.cc:439] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.056118 990 tablet_bootstrap.cc:590] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.057356 988 tablet_bootstrap.cc:590] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.058447 989 tablet_bootstrap.cc:439] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.058574 989 ts_tablet_manager.cc:1053] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s I0513 22:33:16.059402 990 tablet_bootstrap.cc:439] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.059612 990 ts_tablet_manager.cc:1053] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s I0513 22:33:16.059546 989 raft_consensus.cc:340] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.059818 989 raft_consensus.cc:366] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.059896 989 raft_consensus.cc:705] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.060124 988 tablet_bootstrap.cc:439] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.060228 988 ts_tablet_manager.cc:1053] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.001s I0513 22:33:16.060122 989 consensus_queue.cc:229] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.060308 990 raft_consensus.cc:340] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.060518 990 raft_consensus.cc:366] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.060575 990 raft_consensus.cc:705] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.060864 988 raft_consensus.cc:340] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.060861 990 consensus_queue.cc:229] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.061076 988 raft_consensus.cc:366] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.061143 988 raft_consensus.cc:705] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.061264 988 consensus_queue.cc:229] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.061905 988 ts_tablet_manager.cc:1081] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0513 22:33:16.062121 988 tablet_bootstrap.cc:439] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.063308 988 tablet_bootstrap.cc:590] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.063551 989 ts_tablet_manager.cc:1081] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s I0513 22:33:16.063771 990 ts_tablet_manager.cc:1081] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s I0513 22:33:16.063817 989 tablet_bootstrap.cc:439] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.064026 990 tablet_bootstrap.cc:439] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.065438 988 tablet_bootstrap.cc:439] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.065563 988 ts_tablet_manager.cc:1053] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s I0513 22:33:16.066499 989 tablet_bootstrap.cc:590] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.067275 990 tablet_bootstrap.cc:590] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.067713 988 raft_consensus.cc:340] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.068042 988 raft_consensus.cc:366] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.068114 988 raft_consensus.cc:705] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.068472 989 tablet_bootstrap.cc:439] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.068444 988 consensus_queue.cc:229] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.068920 989 ts_tablet_manager.cc:1053] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s I0513 22:33:16.069120 988 ts_tablet_manager.cc:1081] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s I0513 22:33:16.069416 988 tablet_bootstrap.cc:439] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.069687 990 tablet_bootstrap.cc:439] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.069823 990 ts_tablet_manager.cc:1053] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s I0513 22:33:16.070032 989 raft_consensus.cc:340] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.070361 989 raft_consensus.cc:366] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.070446 989 raft_consensus.cc:705] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.070528 990 raft_consensus.cc:340] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.070716 990 raft_consensus.cc:366] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.070775 990 raft_consensus.cc:705] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.070930 989 consensus_queue.cc:229] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.070982 990 consensus_queue.cc:229] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.071226 988 tablet_bootstrap.cc:590] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.071540 989 ts_tablet_manager.cc:1081] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s I0513 22:33:16.071758 989 tablet_bootstrap.cc:439] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.073490 989 tablet_bootstrap.cc:590] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.074419 990 ts_tablet_manager.cc:1081] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s I0513 22:33:16.074631 990 tablet_bootstrap.cc:439] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.076920 989 tablet_bootstrap.cc:439] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.077044 989 ts_tablet_manager.cc:1053] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s I0513 22:33:16.077069 988 tablet_bootstrap.cc:439] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.077119 990 tablet_bootstrap.cc:590] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.077226 988 ts_tablet_manager.cc:1053] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s I0513 22:33:16.077711 989 raft_consensus.cc:340] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.077879 989 raft_consensus.cc:366] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.077837 988 raft_consensus.cc:340] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.077950 989 raft_consensus.cc:705] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.078094 988 raft_consensus.cc:366] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.078249 988 raft_consensus.cc:705] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.078236 989 consensus_queue.cc:229] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.078410 988 consensus_queue.cc:229] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.079123 990 tablet_bootstrap.cc:439] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.079152 988 ts_tablet_manager.cc:1081] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s I0513 22:33:16.079403 988 tablet_bootstrap.cc:439] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.079346 990 ts_tablet_manager.cc:1053] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s I0513 22:33:16.079824 989 ts_tablet_manager.cc:1081] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.003s user 0.001s sys 0.001s I0513 22:33:16.080034 989 tablet_bootstrap.cc:439] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.080241 990 raft_consensus.cc:340] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.080574 990 raft_consensus.cc:366] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.080685 990 raft_consensus.cc:705] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.080940 990 consensus_queue.cc:229] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.081655 989 tablet_bootstrap.cc:590] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.082681 988 tablet_bootstrap.cc:590] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.083003 990 ts_tablet_manager.cc:1081] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s I0513 22:33:16.083274 990 tablet_bootstrap.cc:439] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.085409 993 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.085636 988 tablet_bootstrap.cc:439] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.085670 989 tablet_bootstrap.cc:439] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.085558 993 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.085822 988 ts_tablet_manager.cc:1053] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s I0513 22:33:16.085836 989 ts_tablet_manager.cc:1053] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.003s I0513 22:33:16.086318 990 tablet_bootstrap.cc:590] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.087085 988 raft_consensus.cc:340] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.086534 989 raft_consensus.cc:340] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.093699 997 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.094117 993 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317) I0513 22:33:16.094008 997 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.094710 988 raft_consensus.cc:366] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.094786 988 raft_consensus.cc:705] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.095441 993 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.101003 989 raft_consensus.cc:366] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.095902 993 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.103987 989 raft_consensus.cc:705] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.104471 993 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) I0513 22:33:16.104792 997 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.104476 988 consensus_queue.cc:229] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.111255 995 raft_consensus.cc:472] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.116288 995 raft_consensus.cc:494] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.119557 995 leader_election.cc:296] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.130303 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.130601 808 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.130940 808 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 0. I0513 22:33:16.131634 908 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.130846 809 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 0. I0513 22:33:16.132128 993 raft_consensus.cc:2640] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.132176 993 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.132441 993 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.134891 908 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.135120 1016 raft_consensus.cc:2640] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.135180 1016 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.135246 1016 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.137104 1016 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.137670 1016 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317) I0513 22:33:16.138167 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" I0513 22:33:16.138286 809 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.139111 993 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.140379 809 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.140663 908 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.140851 1016 raft_consensus.cc:2640] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.105319 989 consensus_queue.cc:229] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.146987 993 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Requested vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) I0513 22:33:16.150771 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" I0513 22:33:16.151167 809 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.153141 988 ts_tablet_manager.cc:1081] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.067s user 0.000s sys 0.002s I0513 22:33:16.154537 988 tablet_bootstrap.cc:439] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.162233 1016 raft_consensus.cc:667] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Leader. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.163477 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" is_pre_election: true I0513 22:33:16.165468 990 tablet_bootstrap.cc:439] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.165050 944 raft_consensus.cc:2346] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 0. I0513 22:33:16.164067 1016 consensus_queue.cc:206] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.167116 989 ts_tablet_manager.cc:1081] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.081s user 0.001s sys 0.008s I0513 22:33:16.167587 809 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.167961 908 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.169554 990 ts_tablet_manager.cc:1053] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.082s user 0.009s sys 0.032s I0513 22:33:16.172664 841 leader_election.cc:310] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.173606 989 tablet_bootstrap.cc:439] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.174484 1023 raft_consensus.cc:2640] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.174851 995 raft_consensus.cc:2640] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.174412 997 raft_consensus.cc:472] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.175335 993 raft_consensus.cc:472] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.175930 995 raft_consensus.cc:472] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.176028 995 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.176416 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.176853 997 raft_consensus.cc:494] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.176595 1023 raft_consensus.cc:667] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Leader. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.178447 997 leader_election.cc:296] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.178645 993 raft_consensus.cc:494] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.179242 809 raft_consensus.cc:2346] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 0. I0513 22:33:16.185375 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.188730 993 leader_election.cc:296] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317) I0513 22:33:16.188393 875 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.185420 874 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.179648 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" is_pre_election: true I0513 22:33:16.191879 874 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.191931 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.189451 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" is_pre_election: true I0513 22:33:16.193744 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" is_pre_election: true I0513 22:33:16.194471 990 raft_consensus.cc:340] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.196678 990 raft_consensus.cc:366] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.191804 873 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" is_pre_election: true I0513 22:33:16.197062 873 raft_consensus.cc:2346] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 0. I0513 22:33:16.197820 1016 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.190554 1023 consensus_queue.cc:206] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.196089 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "a40718aa091247438d353d8c7739041e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" is_pre_election: true W0513 22:33:16.199666 908 leader_election.cc:348] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317): Illegal state: must be running to vote when last-logged opid is not known I0513 22:33:16.199793 943 raft_consensus.cc:2346] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 0. I0513 22:33:16.200734 988 tablet_bootstrap.cc:590] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.200655 995 raft_consensus.cc:494] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.202597 990 raft_consensus.cc:705] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.200294 1016 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.203131 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" I0513 22:33:16.203397 995 leader_election.cc:296] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 election: Requested vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.197994 723 catalog_manager.cc:3988] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 reported cstate change: term changed from 0 to 1, leader changed from to fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131). New cstate: current_term: 1 leader_uuid: "fb91e39117a5489e85c5a494ee14bcc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: HEALTHY } } } I0513 22:33:16.204059 773 leader_election.cc:310] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.204989 989 tablet_bootstrap.cc:590] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.205034 907 leader_election.cc:310] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3; no voters: 6fe2349260fd4c51a94a6bc60fa6043b I0513 22:33:16.206300 809 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.203891 990 consensus_queue.cc:229] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.207324 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.207537 1016 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) I0513 22:33:16.207955 997 raft_consensus.cc:2640] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.208032 997 raft_consensus.cc:472] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.208019 808 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.208525 808 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 0. I0513 22:33:16.208619 997 raft_consensus.cc:2886] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.208735 943 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.209866 873 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" is_pre_election: true I0513 22:33:16.210992 993 raft_consensus.cc:2640] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.211068 993 raft_consensus.cc:472] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.207481 1027 raft_consensus.cc:472] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.210651 908 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.214275 993 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 W0513 22:33:16.215034 907 leader_election.cc:348] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393): Illegal state: must be running to vote when last-logged opid is not known I0513 22:33:16.215446 1016 raft_consensus.cc:2640] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.216111 875 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.217306 877 raft_consensus.cc:2321] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb91e39117a5489e85c5a494ee14bcc3 for term 1 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.219446 943 raft_consensus.cc:2346] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 1. I0513 22:33:16.219446 874 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.219574 876 raft_consensus.cc:2321] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate fb91e39117a5489e85c5a494ee14bcc3 for term 1 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.219795 1016 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.221508 809 raft_consensus.cc:2346] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 1. I0513 22:33:16.223237 993 raft_consensus.cc:494] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.223835 993 leader_election.cc:296] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317) I0513 22:33:16.224083 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.224201 877 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.226447 877 raft_consensus.cc:2346] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.227108 907 leader_election.cc:310] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.227588 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" W0513 22:33:16.228054 908 leader_election.cc:348] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317): Illegal state: must be running to vote when last-logged opid is not known I0513 22:33:16.228693 997 raft_consensus.cc:494] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.229341 997 leader_election.cc:296] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.229523 993 raft_consensus.cc:2640] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.229863 993 raft_consensus.cc:667] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Leader. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.230099 841 leader_election.cc:310] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.230548 995 raft_consensus.cc:2640] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.231649 944 raft_consensus.cc:2307] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 1 because replica is either leader or believes a valid leader to be alive. I0513 22:33:16.232019 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "a40718aa091247438d353d8c7739041e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.239218 943 raft_consensus.cc:2886] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.239346 1016 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.211436 1027 raft_consensus.cc:494] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.232412 993 consensus_queue.cc:206] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.240768 725 catalog_manager.cc:3988] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 reported cstate change: term changed from 0 to 1, leader changed from to fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131). New cstate: current_term: 1 leader_uuid: "fb91e39117a5489e85c5a494ee14bcc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: HEALTHY } } } I0513 22:33:16.243958 995 raft_consensus.cc:667] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Becoming Leader. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Running, Role: LEADER I0513 22:33:16.251803 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.257403 943 raft_consensus.cc:2346] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 1. I0513 22:33:16.257865 773 leader_election.cc:310] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.258486 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" is_pre_election: true I0513 22:33:16.258781 990 ts_tablet_manager.cc:1081] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.084s user 0.008s sys 0.024s I0513 22:33:16.258855 874 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "a40718aa091247438d353d8c7739041e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.259325 990 tablet_bootstrap.cc:439] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.259332 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" is_pre_election: true I0513 22:33:16.259500 944 raft_consensus.cc:2346] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 0. I0513 22:33:16.260037 1027 leader_election.cc:296] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.260648 841 leader_election.cc:310] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.260944 1027 raft_consensus.cc:2640] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.255234 995 consensus_queue.cc:206] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.261240 988 tablet_bootstrap.cc:439] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.261363 988 ts_tablet_manager.cc:1053] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.108s user 0.007s sys 0.017s I0513 22:33:16.261335 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "a40718aa091247438d353d8c7739041e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" is_pre_election: true I0513 22:33:16.261286 725 catalog_manager.cc:3988] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 reported cstate change: term changed from 0 to 1, leader changed from to fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131). New cstate: current_term: 1 leader_uuid: "fb91e39117a5489e85c5a494ee14bcc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: HEALTHY } } } I0513 22:33:16.261710 877 raft_consensus.cc:2276] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b in current term 1: Already voted for candidate fb91e39117a5489e85c5a494ee14bcc3 in this term. I0513 22:33:16.262054 809 raft_consensus.cc:2346] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 0. W0513 22:33:16.262243 772 leader_election.cc:348] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393): Illegal state: must be running to vote when last-logged opid is not known W0513 22:33:16.262603 772 leader_election.cc:348] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393): Illegal state: must be running to vote when last-logged opid is not known I0513 22:33:16.262517 988 raft_consensus.cc:340] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.262711 988 raft_consensus.cc:366] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.262953 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.263197 997 raft_consensus.cc:2588] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I0513 22:33:16.262764 988 raft_consensus.cc:705] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.263290 1029 raft_consensus.cc:2640] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.264505 1029 raft_consensus.cc:667] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Becoming Leader. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Running, Role: LEADER I0513 22:33:16.264801 990 tablet_bootstrap.cc:590] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.265116 1027 raft_consensus.cc:472] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.265194 989 tablet_bootstrap.cc:439] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.264932 1016 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.265440 988 consensus_queue.cc:229] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.265774 1027 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.265874 989 ts_tablet_manager.cc:1053] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.098s user 0.006s sys 0.011s I0513 22:33:16.266917 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" I0513 22:33:16.267560 1016 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Requested vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) I0513 22:33:16.267745 988 ts_tablet_manager.cc:1081] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s I0513 22:33:16.267848 809 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.267117 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.269179 877 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.269269 988 tablet_bootstrap.cc:439] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b: Bootstrap starting. I0513 22:33:16.269757 725 catalog_manager.cc:3988] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b reported cstate change: term changed from 0 to 1, leader changed from to 75a5cb20b56443639cfe49c47274d76b (127.0.116.130). New cstate: current_term: 1 leader_uuid: "75a5cb20b56443639cfe49c47274d76b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: UNKNOWN } } } I0513 22:33:16.266659 1029 consensus_queue.cc:206] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.272364 989 raft_consensus.cc:340] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.273952 989 raft_consensus.cc:366] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.274785 989 raft_consensus.cc:705] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.275434 989 consensus_queue.cc:229] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.275920 877 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.276926 989 ts_tablet_manager.cc:1081] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.010s user 0.001s sys 0.001s I0513 22:33:16.276926 907 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.277200 989 tablet_bootstrap.cc:439] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3: Bootstrap starting. I0513 22:33:16.278012 809 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.279289 1016 raft_consensus.cc:2640] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.279678 1016 raft_consensus.cc:667] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Leader. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.280747 1016 consensus_queue.cc:206] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.281401 1027 raft_consensus.cc:494] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.281913 988 tablet_bootstrap.cc:590] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b: No blocks or log segments found. Creating new log. I0513 22:33:16.282272 1027 leader_election.cc:296] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 election: Requested vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.282785 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.283069 944 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.283391 990 tablet_bootstrap.cc:439] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.283089 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" I0513 22:33:16.284662 990 ts_tablet_manager.cc:1053] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.025s user 0.005s sys 0.004s I0513 22:33:16.284946 809 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.285310 725 catalog_manager.cc:3988] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b reported cstate change: term changed from 0 to 1, leader changed from to 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129). New cstate: current_term: 1 leader_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: UNKNOWN } } } I0513 22:33:16.287873 944 raft_consensus.cc:2346] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 1. I0513 22:33:16.288373 841 leader_election.cc:310] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.288597 809 raft_consensus.cc:2346] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 1. I0513 22:33:16.289232 990 raft_consensus.cc:340] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.289525 1027 raft_consensus.cc:2640] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.289839 990 raft_consensus.cc:366] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.289914 1027 raft_consensus.cc:667] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Becoming Leader. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Running, Role: LEADER I0513 22:33:16.290922 1027 consensus_queue.cc:206] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.291858 990 raft_consensus.cc:705] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.292238 990 consensus_queue.cc:229] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.293202 989 tablet_bootstrap.cc:590] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3: No blocks or log segments found. Creating new log. I0513 22:33:16.293778 990 ts_tablet_manager.cc:1081] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s I0513 22:33:16.294361 988 tablet_bootstrap.cc:439] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b: No bootstrap required, opened a new log I0513 22:33:16.294670 723 catalog_manager.cc:3988] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b reported cstate change: term changed from 0 to 1, leader changed from to 75a5cb20b56443639cfe49c47274d76b (127.0.116.130). New cstate: current_term: 1 leader_uuid: "75a5cb20b56443639cfe49c47274d76b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: UNKNOWN } } } I0513 22:33:16.295670 725 catalog_manager.cc:3988] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 reported cstate change: term changed from 0 to 1, leader changed from to fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131). New cstate: current_term: 1 leader_uuid: "fb91e39117a5489e85c5a494ee14bcc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: HEALTHY } } } I0513 22:33:16.296063 990 tablet_bootstrap.cc:439] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.298720 988 ts_tablet_manager.cc:1053] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b: Time spent bootstrapping tablet: real 0.025s user 0.002s sys 0.004s I0513 22:33:16.300176 990 tablet_bootstrap.cc:590] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.300546 988 raft_consensus.cc:340] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.300812 988 raft_consensus.cc:366] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.300866 988 raft_consensus.cc:705] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Initialized, Role: FOLLOWER I0513 22:33:16.301141 989 tablet_bootstrap.cc:439] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3: No bootstrap required, opened a new log I0513 22:33:16.301143 988 consensus_queue.cc:229] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.301386 989 ts_tablet_manager.cc:1053] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.001s I0513 22:33:16.301723 988 ts_tablet_manager.cc:1081] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s I0513 22:33:16.301818 990 tablet_bootstrap.cc:439] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.302006 990 ts_tablet_manager.cc:1053] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s I0513 22:33:16.302083 989 raft_consensus.cc:340] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.302306 989 raft_consensus.cc:366] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.302353 989 raft_consensus.cc:705] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Initialized, Role: FOLLOWER I0513 22:33:16.302517 989 consensus_queue.cc:229] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.302603 990 raft_consensus.cc:340] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.302778 990 raft_consensus.cc:366] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.302832 990 raft_consensus.cc:705] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.303011 990 consensus_queue.cc:229] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.303192 989 ts_tablet_manager.cc:1081] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0513 22:33:16.303715 990 ts_tablet_manager.cc:1081] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s I0513 22:33:16.304014 990 tablet_bootstrap.cc:439] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: Bootstrap starting. I0513 22:33:16.305202 990 tablet_bootstrap.cc:590] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: No blocks or log segments found. Creating new log. I0513 22:33:16.306632 990 tablet_bootstrap.cc:439] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: No bootstrap required, opened a new log I0513 22:33:16.306787 990 ts_tablet_manager.cc:1053] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s I0513 22:33:16.306962 1027 raft_consensus.cc:472] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.307085 1027 raft_consensus.cc:494] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.308210 990 raft_consensus.cc:340] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.308347 1027 leader_election.cc:296] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.308404 990 raft_consensus.cc:366] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I0513 22:33:16.308444 990 raft_consensus.cc:705] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Initialized, Role: FOLLOWER I0513 22:33:16.308601 990 consensus_queue.cc:229] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.308802 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.308890 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" is_pre_election: true I0513 22:33:16.309013 809 raft_consensus.cc:2346] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 0. I0513 22:33:16.309039 990 ts_tablet_manager.cc:1081] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s I0513 22:33:16.309082 944 raft_consensus.cc:2346] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 0. I0513 22:33:16.309314 841 leader_election.cc:310] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.309643 1027 raft_consensus.cc:2640] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.309700 1027 raft_consensus.cc:472] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.309744 1027 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.311308 1027 raft_consensus.cc:494] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.311784 1027 leader_election.cc:296] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 election: Requested vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.312116 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" I0513 22:33:16.312212 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.312291 809 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.312422 944 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.314541 809 raft_consensus.cc:2346] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 1. I0513 22:33:16.314857 944 raft_consensus.cc:2346] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75a5cb20b56443639cfe49c47274d76b in term 1. I0513 22:33:16.315006 841 leader_election.cc:310] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.315263 1027 raft_consensus.cc:2640] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.315456 1027 raft_consensus.cc:667] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Becoming Leader. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Running, Role: LEADER I0513 22:33:16.315646 1027 consensus_queue.cc:206] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.317498 722 catalog_manager.cc:3988] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b reported cstate change: term changed from 0 to 1, leader changed from to 75a5cb20b56443639cfe49c47274d76b (127.0.116.130). New cstate: current_term: 1 leader_uuid: "75a5cb20b56443639cfe49c47274d76b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: UNKNOWN } } } I0513 22:33:16.382858 1016 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.383070 1016 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.383680 1016 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) I0513 22:33:16.384253 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" is_pre_election: true I0513 22:33:16.384282 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.384483 809 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 0. I0513 22:33:16.384482 877 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 0. I0513 22:33:16.384817 908 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.385071 1016 raft_consensus.cc:2640] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.385123 1016 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.385165 1016 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.386690 1016 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.387140 1016 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Requested vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) I0513 22:33:16.387562 809 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" I0513 22:33:16.387631 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "fb91e39117a5489e85c5a494ee14bcc3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.387753 809 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.387753 877 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.389840 877 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.389827 809 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb91e39117a5489e85c5a494ee14bcc3 in term 1. I0513 22:33:16.390156 908 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.390408 1016 raft_consensus.cc:2640] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.390568 1016 raft_consensus.cc:667] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Leader. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.390748 1016 consensus_queue.cc:206] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.392603 722 catalog_manager.cc:3988] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 reported cstate change: term changed from 0 to 1, leader changed from to fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131). New cstate: current_term: 1 leader_uuid: "fb91e39117a5489e85c5a494ee14bcc3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: HEALTHY } } } I0513 22:33:16.405498 1029 raft_consensus.cc:472] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.405656 1029 raft_consensus.cc:494] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.406199 1029 leader_election.cc:296] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.406626 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" is_pre_election: true I0513 22:33:16.406625 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" is_pre_election: true I0513 22:33:16.407519 944 raft_consensus.cc:2346] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 0. I0513 22:33:16.407580 877 raft_consensus.cc:2346] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 0. I0513 22:33:16.407832 773 leader_election.cc:310] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.408144 1029 raft_consensus.cc:2640] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Leader pre-election won for term 1 I0513 22:33:16.408223 1029 raft_consensus.cc:472] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I0513 22:33:16.408277 1029 raft_consensus.cc:2886] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.410017 1029 raft_consensus.cc:494] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.410622 1029 leader_election.cc:296] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.411046 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.411051 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.411207 944 raft_consensus.cc:2886] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3 [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.411255 877 raft_consensus.cc:2886] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.412905 944 raft_consensus.cc:2346] T c1cbbc2b2f22460aa0617e7a5c823a57 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 1. I0513 22:33:16.413190 877 raft_consensus.cc:2346] T c1cbbc2b2f22460aa0617e7a5c823a57 P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 1. I0513 22:33:16.413265 773 leader_election.cc:310] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.413522 1029 raft_consensus.cc:2640] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Leader election won for term 1 I0513 22:33:16.413642 1029 raft_consensus.cc:667] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Becoming Leader. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Running, Role: LEADER I0513 22:33:16.413830 1029 consensus_queue.cc:206] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.417708 722 catalog_manager.cc:3988] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b reported cstate change: term changed from 0 to 1, leader changed from to 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129). New cstate: current_term: 1 leader_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } health_report { overall_health: UNKNOWN } } } I0513 22:33:16.462265 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.462272 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.462610 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.462761 808 raft_consensus.cc:472] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.462738 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.462910 808 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.462929 806 raft_consensus.cc:472] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.463114 805 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.463845 809 raft_consensus.cc:455] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.462437 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.463811 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.464043 804 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.464224 804 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.464303 807 raft_consensus.cc:472] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.464484 805 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.464509 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.464776 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.464929 803 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.464995 799 tablet_service.cc:1057] Processing DeleteTablet for tablet c1cbbc2b2f22460aa0617e7a5c823a57 with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.464962 801 raft_consensus.cc:472] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.465530 801 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.465646 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.465505 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.467568 806 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.467725 807 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.468046 803 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.467537 802 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.468319 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.468478 800 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.470077 809 raft_consensus.cc:455] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.470393 808 raft_consensus.cc:494] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.470791 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.471128 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.471256 800 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.471397 802 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.473067 1057 tablet_replica.cc:264] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: stopping tablet replica I0513 22:33:16.473122 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.473245 800 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.473647 1057 raft_consensus.cc:2143] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Raft consensus shutting down. I0513 22:33:16.473654 877 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.473817 877 raft_consensus.cc:2881] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Stepping down as leader of term 1 I0513 22:33:16.486735 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.487939 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.488993 800 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.488996 877 raft_consensus.cc:705] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Running, Role: LEADER I0513 22:33:16.489014 944 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.488760 808 leader_election.cc:296] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.489970 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.489409 807 raft_consensus.cc:494] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.490504 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.491098 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.491247 800 raft_consensus.cc:472] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.491961 800 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.493453 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.493727 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.493867 943 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.493849 805 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.491961 877 consensus_queue.cc:229] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.494635 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.497560 807 leader_election.cc:296] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.497582 876 raft_consensus.cc:2881] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Stepping down as leader of term 1 I0513 22:33:16.497864 1057 pending_rounds.cc:64] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: Trying to abort 1 pending transactions. I0513 22:33:16.497984 876 raft_consensus.cc:705] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Running, Role: LEADER I0513 22:33:16.498096 877 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.497372 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.495482 803 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.498757 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.496650 801 raft_consensus.cc:494] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.500113 809 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.498023 1057 pending_rounds.cc:70] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: Aborting transaction as it isn't in flight: id { term: 1 index: 1 } timestamp: 6380694718112780288 op_type: NO_OP noop_request { } I0513 22:33:16.496654 802 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.501672 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.501884 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.502476 874 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.502483 873 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.502699 874 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.502955 805 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.502741 876 consensus_queue.cc:229] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.503302 876 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.503747 943 raft_consensus.cc:2346] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 2. I0513 22:33:16.502516 942 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.504319 942 raft_consensus.cc:2881] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Stepping down as leader of term 1 I0513 22:33:16.504649 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.500020 804 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.504395 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.504861 941 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.505285 939 raft_consensus.cc:2881] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Stepping down as leader of term 1 I0513 22:33:16.503403 800 raft_consensus.cc:494] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.505460 941 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.504650 806 raft_consensus.cc:494] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.507205 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.505740 801 leader_election.cc:296] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.505995 803 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.509089 875 raft_consensus.cc:2881] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Stepping down as leader of term 1 I0513 22:33:16.509029 809 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.509557 875 raft_consensus.cc:705] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 1 LEADER]: Becoming Follower/Learner. State: Replica: 75a5cb20b56443639cfe49c47274d76b, State: Running, Role: LEADER I0513 22:33:16.510094 877 raft_consensus.cc:2293] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.509938 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.511195 944 raft_consensus.cc:2346] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 2. I0513 22:33:16.511641 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.511762 808 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.510993 1057 raft_consensus.cc:2723] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: NO_OP replication failed: Aborted: Transaction aborted I0513 22:33:16.512380 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.509577 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.510159 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.513327 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.513360 942 raft_consensus.cc:705] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.511690 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.514328 872 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.514566 940 raft_consensus.cc:2881] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Stepping down as leader of term 1 I0513 22:33:16.513540 1057 raft_consensus.cc:2170] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Raft consensus is shut down! I0513 22:33:16.514894 773 leader_election.cc:310] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.514390 871 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.515816 773 leader_election.cc:310] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: 75a5cb20b56443639cfe49c47274d76b I0513 22:33:16.516202 939 raft_consensus.cc:705] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.516222 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.511417 875 consensus_queue.cc:229] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.516940 875 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.515816 942 consensus_queue.cc:229] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.516999 804 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.517132 939 consensus_queue.cc:229] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.516374 870 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.517437 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.517846 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.518353 873 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.519402 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.518579 802 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.519896 807 raft_consensus.cc:472] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.519982 807 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.521129 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.522419 809 raft_consensus.cc:472] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.522393 800 leader_election.cc:296] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.522895 808 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.522933 874 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 2. I0513 22:33:16.523272 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.523815 873 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 2. I0513 22:33:16.524111 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.525369 803 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.525954 876 raft_consensus.cc:2293] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.525457 806 leader_election.cc:296] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.526464 940 raft_consensus.cc:705] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.526639 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.527235 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.527362 805 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.527402 997 raft_consensus.cc:2603] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won I0513 22:33:16.526821 869 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.527910 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.528064 869 raft_consensus.cc:2276] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b in current term 1: Already voted for candidate fb91e39117a5489e85c5a494ee14bcc3 in this term. I0513 22:33:16.528368 805 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.528715 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.527778 940 consensus_queue.cc:229] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.528686 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.529211 938 raft_consensus.cc:2276] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b in current term 1: Already voted for candidate fb91e39117a5489e85c5a494ee14bcc3 in this term. I0513 22:33:16.530200 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.530742 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.531029 942 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.531261 943 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.531445 944 raft_consensus.cc:2881] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Stepping down as leader of term 1 I0513 22:33:16.531571 808 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.531271 939 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.533150 804 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.533197 872 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 2. I0513 22:33:16.533246 804 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.534387 941 raft_consensus.cc:2346] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 2. I0513 22:33:16.535051 805 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.535297 875 raft_consensus.cc:2293] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.535357 997 raft_consensus.cc:2603] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won I0513 22:33:16.535323 807 raft_consensus.cc:494] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.535642 1029 raft_consensus.cc:2603] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won I0513 22:33:16.535742 942 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.535827 807 leader_election.cc:296] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.535902 943 raft_consensus.cc:2346] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 3. I0513 22:33:16.536669 805 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.536706 870 raft_consensus.cc:2293] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.537683 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.537806 939 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.538053 1057 ts_tablet_manager.cc:1377] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting tablet data with delete state TABLET_DATA_DELETED I0513 22:33:16.538951 800 raft_consensus.cc:472] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.539057 800 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.539053 940 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.539090 801 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.539124 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.539800 772 leader_election.cc:310] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.540346 1029 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won I0513 22:33:16.540508 806 raft_consensus.cc:472] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.540491 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.540513 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.540659 806 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.540756 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.541038 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.541478 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.542163 944 raft_consensus.cc:705] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.542590 808 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.542726 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.543066 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.543440 802 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.543710 773 leader_election.cc:310] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.544503 871 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 2. I0513 22:33:16.544633 773 leader_election.cc:310] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.544723 940 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.544682 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.544761 802 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.543664 944 consensus_queue.cc:229] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.545333 938 raft_consensus.cc:2321] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.545925 809 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.546196 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.546356 807 raft_consensus.cc:472] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.546613 800 raft_consensus.cc:494] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.547271 800 leader_election.cc:296] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.547497 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.548359 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.548503 939 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.549234 872 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.549937 875 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.549311 804 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.550691 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.549232 809 raft_consensus.cc:494] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.551862 944 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.552662 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.552908 804 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.553176 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.553444 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.554071 803 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.554262 1029 raft_consensus.cc:2603] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won I0513 22:33:16.554571 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.554708 938 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.554685 1029 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won I0513 22:33:16.558470 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.558851 875 raft_consensus.cc:2293] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.559603 805 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.559703 804 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.560073 807 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.560600 805 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.560829 801 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.561699 802 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.562376 944 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.562389 870 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.562865 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.563339 943 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.563436 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.563647 802 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.563807 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.564250 872 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 3. I0513 22:33:16.564729 1058 raft_consensus.cc:2603] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won I0513 22:33:16.565441 800 raft_consensus.cc:472] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.565492 1027 raft_consensus.cc:472] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I0513 22:33:16.565794 1027 raft_consensus.cc:494] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.566378 1057 ts_tablet_manager.cc:1390] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.1 I0513 22:33:16.567167 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.567564 808 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "a40718aa091247438d353d8c7739041e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" is_pre_election: true I0513 22:33:16.567876 871 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.568480 869 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.568603 869 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.568574 942 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.568601 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.568738 942 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.568771 940 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.567785 809 leader_election.cc:296] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.569553 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.571346 803 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.571990 939 raft_consensus.cc:2346] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 4. I0513 22:33:16.573313 806 raft_consensus.cc:494] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.571238 941 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "a40718aa091247438d353d8c7739041e" candidate_uuid: "75a5cb20b56443639cfe49c47274d76b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" is_pre_election: true I0513 22:33:16.574156 871 raft_consensus.cc:2293] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 4 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.574154 806 leader_election.cc:296] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.575644 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.575673 803 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.575794 875 raft_consensus.cc:2886] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.576238 869 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 3. I0513 22:33:16.576745 805 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.577939 876 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.578418 870 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 3. I0513 22:33:16.578410 807 raft_consensus.cc:494] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.578830 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.579113 801 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.577415 1059 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won I0513 22:33:16.579670 802 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.580153 808 raft_consensus.cc:2307] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate 75a5cb20b56443639cfe49c47274d76b for term 1 because replica is either leader or believes a valid leader to be alive. I0513 22:33:16.581472 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.582011 800 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.582299 805 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.582495 1027 leader_election.cc:296] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129:34317), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.583659 876 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 3. I0513 22:33:16.584264 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.584365 1057 log.cc:990] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/wals/c1cbbc2b2f22460aa0617e7a5c823a57 I0513 22:33:16.584528 938 raft_consensus.cc:2346] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 3. I0513 22:33:16.584774 773 leader_election.cc:310] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.584985 1057 ts_tablet_manager.cc:1411] T c1cbbc2b2f22460aa0617e7a5c823a57 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting consensus metadata I0513 22:33:16.585690 773 leader_election.cc:310] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: 75a5cb20b56443639cfe49c47274d76b I0513 22:33:16.586051 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.586241 997 raft_consensus.cc:2588] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority I0513 22:33:16.586668 941 raft_consensus.cc:2276] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 75a5cb20b56443639cfe49c47274d76b in current term 1: Already voted for candidate 6fe2349260fd4c51a94a6bc60fa6043b in this term. I0513 22:33:16.585886 800 raft_consensus.cc:494] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.586998 804 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.587294 1059 raft_consensus.cc:2603] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won I0513 22:33:16.587265 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.587255 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.587518 1058 raft_consensus.cc:2603] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won I0513 22:33:16.587656 941 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.587630 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.587846 941 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Advancing to term 4 I0513 22:33:16.587949 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.588294 943 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.588196 937 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.589279 937 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.589326 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.589664 799 tablet_service.cc:1057] Processing DeleteTablet for tablet 73f1725c5aa04760a9d5e43d09f66e9e with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.589581 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.590821 803 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.590859 802 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.591739 942 raft_consensus.cc:2346] T 8002a0fb94514f77a8601f629424436e P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 3. I0513 22:33:16.591796 941 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 4 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.592167 807 leader_election.cc:296] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.592186 773 leader_election.cc:310] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.592602 937 raft_consensus.cc:2346] T 01c2af1f12034771ba026a1fac9203df P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 4. I0513 22:33:16.592869 940 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.592924 773 leader_election.cc:310] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.593250 939 raft_consensus.cc:2321] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 4 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.593700 801 raft_consensus.cc:472] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.593792 801 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.593839 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.594267 841 leader_election.cc:310] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 75a5cb20b56443639cfe49c47274d76b; no voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.594704 1027 raft_consensus.cc:2886] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [term 0 FOLLOWER]: Advancing to term 1 I0513 22:33:16.594579 804 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.594846 802 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.595099 938 raft_consensus.cc:2881] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Stepping down as leader of term 1 I0513 22:33:16.595180 938 raft_consensus.cc:705] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 LEADER]: Becoming Follower/Learner. State: Replica: fb91e39117a5489e85c5a494ee14bcc3, State: Running, Role: LEADER I0513 22:33:16.595443 802 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.595522 938 consensus_queue.cc:229] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 1.1, Last appended by leader: 1, Current term: 1, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.596767 938 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.597295 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.596441 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.597980 1027 raft_consensus.cc:2588] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I0513 22:33:16.596599 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.598704 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.599079 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.599046 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.599364 997 raft_consensus.cc:2603] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won I0513 22:33:16.599894 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.600476 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.600744 807 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.600924 807 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.600915 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 2 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.600994 943 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.601122 875 raft_consensus.cc:2293] T 8002a0fb94514f77a8601f629424436e P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.601342 876 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Advancing to term 2 I0513 22:33:16.601030 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.601701 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.602048 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.602030 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.602039 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.602531 871 raft_consensus.cc:2886] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.602511 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.602599 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.602931 869 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.603018 872 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.601884 873 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.603332 873 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.605554 804 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.605576 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.605916 873 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 4. I0513 22:33:16.605357 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.606338 872 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 4. I0513 22:33:16.606895 800 leader_election.cc:296] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.606493 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.607623 871 raft_consensus.cc:2293] T 01c2af1f12034771ba026a1fac9203df P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 4 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.607628 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.608348 939 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Advancing to term 4 I0513 22:33:16.608356 876 raft_consensus.cc:2346] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 2. I0513 22:33:16.608528 997 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won I0513 22:33:16.608649 938 raft_consensus.cc:2293] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 2 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.609216 943 raft_consensus.cc:2346] T 73f1725c5aa04760a9d5e43d09f66e9e P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 5. I0513 22:33:16.609553 773 leader_election.cc:310] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, fb91e39117a5489e85c5a494ee14bcc3; no voters: I0513 22:33:16.609836 997 raft_consensus.cc:2640] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election won for term 5 I0513 22:33:16.610049 939 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 4 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.610376 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.610237 997 raft_consensus.cc:667] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 LEADER]: Becoming Leader. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Running, Role: LEADER I0513 22:33:16.610497 944 raft_consensus.cc:2321] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.610746 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.610744 997 consensus_queue.cc:206] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 5, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.611920 805 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.611922 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 W0513 22:33:16.611901 773 leader_election.cc:405] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Vote denied by peer fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) with higher term. Message: Service unavailable: T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.612841 1059 raft_consensus.cc:2640] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election won for term 3 I0513 22:33:16.616057 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.616062 1059 raft_consensus.cc:667] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Becoming Leader. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Running, Role: LEADER I0513 22:33:16.616312 870 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.616314 875 raft_consensus.cc:2886] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.616380 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.616309 869 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.617583 802 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.617664 802 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.617563 801 raft_consensus.cc:494] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.618358 801 leader_election.cc:296] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.619400 1058 raft_consensus.cc:2640] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Leader election won for term 4 I0513 22:33:16.616516 1059 consensus_queue.cc:206] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.619812 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.619818 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.620182 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.621300 940 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.621680 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.621738 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.622119 876 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.622380 772 leader_election.cc:310] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.624245 869 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 4. I0513 22:33:16.624631 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.624677 809 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.625142 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.626322 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 3 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.627261 1059 raft_consensus.cc:2603] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Leader election decision vote started in defunct term 2: won I0513 22:33:16.627439 944 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 2 FOLLOWER]: Advancing to term 3 I0513 22:33:16.628535 940 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 4 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.628507 1058 raft_consensus.cc:667] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Becoming Leader. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Running, Role: LEADER I0513 22:33:16.629300 875 raft_consensus.cc:2293] T 73f1725c5aa04760a9d5e43d09f66e9e P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 5 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.629640 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.630386 870 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 4. I0513 22:33:16.631208 1057 tablet_replica.cc:264] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: stopping tablet replica I0513 22:33:16.631723 1057 raft_consensus.cc:2143] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 LEADER]: Raft consensus shutting down. I0513 22:33:16.630656 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.632669 805 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.628144 807 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.629848 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.633577 801 raft_consensus.cc:472] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.633973 807 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.633895 722 catalog_manager.cc:3988] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b reported cstate change: term changed from 1 to 5, leader changed from 75a5cb20b56443639cfe49c47274d76b (127.0.116.130) to 6fe2349260fd4c51a94a6bc60fa6043b (127.0.116.129). New cstate: current_term: 5 leader_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } } I0513 22:33:16.634241 876 raft_consensus.cc:2346] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 3. I0513 22:33:16.634243 804 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.634629 804 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.635100 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.635498 772 leader_election.cc:310] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.636368 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.636804 1057 pending_rounds.cc:64] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: Trying to abort 1 pending transactions. I0513 22:33:16.636778 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.636858 1057 pending_rounds.cc:70] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: Aborting transaction as it isn't in flight: id { term: 5 index: 1 } timestamp: 6380694718923833344 op_type: NO_OP noop_request { } I0513 22:33:16.637169 1057 raft_consensus.cc:2723] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 LEADER]: NO_OP replication failed: Aborted: Transaction aborted I0513 22:33:16.637245 1057 raft_consensus.cc:2170] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Raft consensus is shut down! I0513 22:33:16.637243 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.637236 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.637192 806 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.638407 809 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.636179 1058 consensus_queue.cc:206] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 4, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.637977 1060 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 3: won I0513 22:33:16.638968 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.639739 801 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 FOLLOWER]: Advancing to term 4 W0513 22:33:16.639236 722 quorum_util.cc:596] leader peer 6fe2349260fd4c51a94a6bc60fa6043b reported health as UNKNOWN; config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } F0513 22:33:16.640089 722 quorum_util.cc:601] Check failed: false Found non-HEALTHY LEADER *** Check failure stack trace: *** *** Aborted at 1557786796 (unix time) try "date -d @1557786796" if you are using GNU date *** I0513 22:33:16.640161 805 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.640653 940 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.643748 801 raft_consensus.cc:494] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.641069 1065 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won I0513 22:33:16.643085 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.642943 944 raft_consensus.cc:2293] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 3 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.644970 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.641798 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.645135 876 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.643389 802 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } W0513 22:33:16.641710 1062 consensus_queue.cc:1099] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Queue is closed or peer was untracked, disregarding peer response. Response: responder_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" status { last_received { term: 5 index: 1 } last_committed_idx: 0 last_received_current_leader { term: 5 index: 1 } } I0513 22:33:16.646399 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 4 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.647051 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.647450 802 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.647684 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.647781 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.647912 809 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader PC: @ 0x7fa560cfac37 gsignal *** SIGABRT (@0x3e8000001d2) received by PID 466 (TID 0x7fa5527a5700) from PID 466; stack trace: *** I0513 22:33:16.648861 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.649149 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.649181 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.649268 800 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.649454 800 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.649916 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.650249 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.650466 939 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Advancing to term 5 I0513 22:33:16.650501 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.651839 940 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 5 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.652139 1059 raft_consensus.cc:2603] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Leader election decision vote started in defunct term 3: won I0513 22:33:16.652590 939 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 5 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.652679 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.652868 800 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.653132 801 leader_election.cc:296] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.653803 1061 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won I0513 22:33:16.653781 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.653991 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.654415 870 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.654418 943 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.655048 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.655066 869 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.655202 869 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Advancing to term 5 @ 0x7fa568934330 (unknown) at ??:0 I0513 22:33:16.654783 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.659090 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.660039 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.660336 875 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.660526 1057 ts_tablet_manager.cc:1377] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting tablet data with delete state TABLET_DATA_DELETED I0513 22:33:16.661564 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.661886 802 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.662297 876 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 5. I0513 22:33:16.663550 803 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.664219 804 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.665194 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.665374 870 raft_consensus.cc:2346] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 4. I0513 22:33:16.666369 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.666754 804 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.666813 944 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 3 FOLLOWER]: Advancing to term 4 I0513 22:33:16.667338 807 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.667435 1064 raft_consensus.cc:2603] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won I0513 22:33:16.668088 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.669306 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.669387 944 raft_consensus.cc:2293] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 4 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.669418 1029 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 3: won I0513 22:33:16.669584 809 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.669726 809 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.670204 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.670370 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.670507 806 raft_consensus.cc:472] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.670657 806 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.671016 805 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.671232 871 raft_consensus.cc:2321] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 6 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.671192 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.671926 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.672346 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.672983 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.673204 808 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.673876 997 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won I0513 22:33:16.674335 802 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.674358 808 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.674402 803 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.674404 869 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 5. I0513 22:33:16.675112 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.675534 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.675537 809 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.675961 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.676092 803 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.676770 802 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.677036 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.677088 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.677085 806 raft_consensus.cc:494] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.677194 804 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.677651 1057 ts_tablet_manager.cc:1390] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 5.1 I0513 22:33:16.677780 939 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.678227 800 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.678262 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.678989 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.679222 804 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.679219 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.679265 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.680048 939 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 6 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. @ 0x7fa560cfac37 gsignal at ??:0 I0513 22:33:16.682963 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.683189 807 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.683272 807 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.683717 772 leader_election.cc:310] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 4 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.683946 1029 raft_consensus.cc:2603] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Leader election decision vote started in defunct term 4: won I0513 22:33:16.684231 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.684626 808 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.685305 869 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.685510 943 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 5 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.685820 807 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.685848 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.686266 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.686349 807 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.686391 875 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 5. I0513 22:33:16.686821 809 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.686934 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.686859 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 5 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.687443 1064 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won I0513 22:33:16.687497 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.687531 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.687739 802 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.687860 808 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.687992 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.688282 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.688336 1057 log.cc:990] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/wals/73f1725c5aa04760a9d5e43d09f66e9e I0513 22:33:16.688340 938 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.688393 807 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.688444 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.689151 1057 ts_tablet_manager.cc:1411] T 73f1725c5aa04760a9d5e43d09f66e9e P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting consensus metadata @ 0x7fa560cfe028 abort at ??:0 I0513 22:33:16.690536 938 raft_consensus.cc:2293] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 5 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.690975 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.691282 807 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Advancing to term 7 I0513 22:33:16.691417 800 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.691789 800 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Advancing to term 7 I0513 22:33:16.692129 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.692277 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.692659 869 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.692705 871 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.694993 943 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.695142 804 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.695233 940 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.695502 875 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 4 FOLLOWER]: Advancing to term 5 I0513 22:33:16.696849 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.697259 944 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.697793 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.699319 806 leader_election.cc:296] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.699707 875 raft_consensus.cc:2346] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 5. I0513 22:33:16.699892 871 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 6. I0513 22:33:16.700047 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.700301 943 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 6 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.700847 869 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 5. @ 0x7fa564076e09 google::logging_fail() at ??:0 I0513 22:33:16.701452 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.701836 772 leader_election.cc:310] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.702474 940 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 5 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.702621 939 raft_consensus.cc:2321] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 6 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.702801 809 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.703032 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.703143 803 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.703258 809 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Advancing to term 7 I0513 22:33:16.703786 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.704042 799 tablet_service.cc:1057] Processing DeleteTablet for tablet 3c141683c43f4fb78c4489e7904b28e8 with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.704319 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.704335 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.704432 808 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.704797 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.705083 805 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.705075 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.705255 804 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.705605 800 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.706048 997 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Leader election decision vote started in defunct term 5: won I0513 22:33:16.706967 944 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 6 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.707540 800 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.707540 1064 raft_consensus.cc:2603] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Leader election decision vote started in defunct term 5: won I0513 22:33:16.707617 869 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.707692 802 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.707993 869 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.708040 802 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Advancing to term 7 I0513 22:33:16.708003 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.708735 871 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.709399 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.709451 1057 tablet_replica.cc:264] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: stopping tablet replica I0513 22:33:16.709565 803 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.709820 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.710011 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.710036 1029 raft_consensus.cc:2588] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Leader election lost for term 6. Reason: could not achieve majority I0513 22:33:16.710184 806 raft_consensus.cc:472] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.710248 806 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.710623 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.710716 804 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.711109 804 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Advancing to term 8 I0513 22:33:16.711298 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.710846 807 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.712019 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.712134 803 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.713371 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.713384 802 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } @ 0x7fa56407862d google::LogMessage::Fail() at ??:0 I0513 22:33:16.715536 804 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.716114 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.715945 809 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.716370 801 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.716583 807 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.716773 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.716975 809 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.717114 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.717236 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.717310 940 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 6 FOLLOWER]: Advancing to term 7 I0513 22:33:16.717227 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.717988 938 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Advancing to term 7 I0513 22:33:16.718334 1029 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Leader election decision vote started in defunct term 6: won I0513 22:33:16.718984 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 5 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.719482 802 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.720099 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.720526 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.720634 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.720769 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.720938 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.721119 873 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.721237 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.721326 944 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 6 FOLLOWER]: Advancing to term 7 I0513 22:33:16.721395 869 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 6. I0513 22:33:16.720686 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.721822 800 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.721930 800 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Advancing to term 8 I0513 22:33:16.722343 804 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 8 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.722935 805 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.722965 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.724165 871 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 6. I0513 22:33:16.724661 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.724965 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.724948 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.725189 1029 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Leader election decision vote started in defunct term 5: won I0513 22:33:16.725209 803 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.725512 939 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 6 FOLLOWER]: Advancing to term 7 I0513 22:33:16.725519 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.725699 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.725908 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.726052 870 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Advancing to term 7 I0513 22:33:16.726370 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.726498 803 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.726966 872 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Advancing to term 8 I0513 22:33:16.727491 801 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.727666 801 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Advancing to term 8 I0513 22:33:16.728372 944 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 7 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.727854 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.729403 872 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 8. I0513 22:33:16.729686 943 raft_consensus.cc:2321] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 8 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.730693 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.731402 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 7 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.731899 875 raft_consensus.cc:2251] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for earlier term 7. Current term is 8. I0513 22:33:16.732481 876 raft_consensus.cc:2321] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 7 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.731894 806 raft_consensus.cc:494] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } @ 0x7fa56407a64c google::LogMessage::SendToLog() at ??:0 I0513 22:33:16.733243 1057 raft_consensus.cc:2143] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Raft consensus shutting down. I0513 22:33:16.733593 806 leader_election.cc:296] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.734247 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.734405 873 raft_consensus.cc:2321] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 7 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.735194 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.735303 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.735661 940 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 7 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.735689 805 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.735762 805 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.737630 804 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.737892 808 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.737895 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.737979 808 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 7 FOLLOWER]: Advancing to term 8 I0513 22:33:16.738036 803 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.738119 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.738261 938 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 7 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.739058 870 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 7 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 7. I0513 22:33:16.739070 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.739563 943 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.739665 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.739698 943 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.740109 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.740388 1057 raft_consensus.cc:2170] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Raft consensus is shut down! I0513 22:33:16.740409 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.740672 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.740892 1061 raft_consensus.cc:2556] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [term 6 FOLLOWER]: Received election callback for term 5 while not running: Illegal state: RaftConsensus is not running: State = Stopped I0513 22:33:16.741915 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.742411 1057 ts_tablet_manager.cc:1377] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting tablet data with delete state TABLET_DATA_DELETED I0513 22:33:16.742679 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.742285 801 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.743077 1064 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Leader election decision vote started in defunct term 6: won I0513 22:33:16.743499 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.743698 801 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 8 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.744175 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.745707 802 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.745971 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.746415 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 W0513 22:33:16.746529 772 leader_election.cc:405] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Vote denied by peer 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) with higher term. Message: Invalid argument: T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for earlier term 7. Current term is 8. I0513 22:33:16.746690 873 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 6 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.747148 873 raft_consensus.cc:2886] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 5 FOLLOWER]: Advancing to term 6 I0513 22:33:16.747207 870 raft_consensus.cc:1181] T a40718aa091247438d353d8c7739041e P 75a5cb20b56443639cfe49c47274d76b [term 1 FOLLOWER]: Refusing update from remote peer 6fe2349260fd4c51a94a6bc60fa6043b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) @ 0x7fa564078189 google::LogMessage::Flush() at ??:0 I0513 22:33:16.747709 805 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.748016 1064 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Leader election decision vote started in defunct term 8: won I0513 22:33:16.748076 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.748368 809 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.748528 807 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.748555 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.749315 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.749537 938 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Advancing to term 8 I0513 22:33:16.745754 800 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.751642 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.751843 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.751891 802 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.751873 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 9 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.751947 806 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.752002 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.752171 873 raft_consensus.cc:2346] T 3c141683c43f4fb78c4489e7904b28e8 P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 6. I0513 22:33:16.752470 805 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 9 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.752379 808 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.752867 806 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.752856 1029 raft_consensus.cc:2603] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Leader election decision vote started in defunct term 6: won I0513 22:33:16.753237 800 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 8 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.753357 939 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 7 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.753386 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.755021 808 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 8 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.755055 943 raft_consensus.cc:2293] T 3c141683c43f4fb78c4489e7904b28e8 P fb91e39117a5489e85c5a494ee14bcc3 [term 6 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 6 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.755076 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.755504 1057 ts_tablet_manager.cc:1390] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0 I0513 22:33:16.755595 1057 log.cc:990] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/wals/3c141683c43f4fb78c4489e7904b28e8 I0513 22:33:16.755635 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.755782 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.755815 940 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Advancing to term 9 I0513 22:33:16.755990 944 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Advancing to term 8 I0513 22:33:16.756268 938 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 8 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.756531 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.756633 804 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.756500 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.758687 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.759212 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.759810 807 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.759979 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.760083 940 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 9 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.760768 939 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 7 FOLLOWER]: Advancing to term 8 I0513 22:33:16.761108 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.761270 805 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.761768 772 leader_election.cc:408] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Cancelling election due to peer responding with higher term I0513 22:33:16.761828 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 @ 0x7fa56407afdf google::LogMessageFatal::~LogMessageFatal() at ??:0 I0513 22:33:16.761395 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.763484 803 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.763512 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.763654 804 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.764179 803 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.764492 805 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.764883 944 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 8 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.764931 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.767930 939 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 8 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.768694 809 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.768738 1057 ts_tablet_manager.cc:1411] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting consensus metadata I0513 22:33:16.768815 809 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.764050 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.769805 802 raft_consensus.cc:455] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.769803 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.769455 806 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.770141 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.770298 808 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.770413 799 tablet_service.cc:1057] Processing DeleteTablet for tablet 01c2af1f12034771ba026a1fac9203df with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.771220 1057 tablet_replica.cc:264] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: stopping tablet replica I0513 22:33:16.771410 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.772513 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.772720 806 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 9 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.772492 803 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.771956 805 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.773133 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.773182 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 9 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.773286 1064 raft_consensus.cc:2588] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority I0513 22:33:16.773329 939 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.773587 803 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 9 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.774140 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 9 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.774372 805 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 9 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.774410 875 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.774574 875 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 7 FOLLOWER]: Advancing to term 8 I0513 22:33:16.775126 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.775285 873 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 8 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.775416 873 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Advancing to term 8 I0513 22:33:16.775449 800 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.776242 876 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 6 FOLLOWER]: Advancing to term 8 I0513 22:33:16.776767 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.777019 1057 raft_consensus.cc:2143] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: Raft consensus shutting down. I0513 22:33:16.778234 1057 pending_rounds.cc:64] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: Trying to abort 1 pending transactions. I0513 22:33:16.779098 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.779417 876 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 8. I0513 22:33:16.779549 809 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.780225 809 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 10 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.781041 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.781065 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 9 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.781141 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.781191 870 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.781359 802 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.781510 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.781725 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.781771 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 7 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.781972 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 9 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.782089 940 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.782968 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.783794 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.783783 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 9 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.784163 869 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" @ 0x7fa56f5424dd kudu::consensus::ShouldEvictReplica() at ??:0 I0513 22:33:16.785300 805 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.785482 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.785867 1057 pending_rounds.cc:70] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: Aborting transaction as it isn't in flight: id { term: 4 index: 1 } timestamp: 6380694719034691584 op_type: NO_OP noop_request { } I0513 22:33:16.785975 1057 raft_consensus.cc:2723] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 LEADER]: NO_OP replication failed: Aborted: Transaction aborted I0513 22:33:16.786082 1057 raft_consensus.cc:2170] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b [term 4 FOLLOWER]: Raft consensus is shut down! I0513 22:33:16.786211 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.786324 804 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.786639 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.786896 940 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 9 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.787119 1057 ts_tablet_manager.cc:1377] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting tablet data with delete state TABLET_DATA_DELETED I0513 22:33:16.788816 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.788951 804 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.789901 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.789952 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.790093 805 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.791038 939 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 9 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.791091 1029 raft_consensus.cc:2588] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Leader election lost for term 7. Reason: Vote denied by peer 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393) with higher term. Message: Invalid argument: T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for earlier term 7. Current term is 8. I0513 22:33:16.791116 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.791203 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.791388 875 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 8. I0513 22:33:16.791417 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.791846 804 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.791991 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.792043 873 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 8. I0513 22:33:16.792119 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.792462 938 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.792536 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.792680 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.792696 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.792793 1061 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Leader election decision vote started in defunct term 7: won I0513 22:33:16.792857 997 consensus_queue.cc:984] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Connected to new peer: Peer: permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.010s I0513 22:33:16.792989 808 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.792990 800 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.793457 944 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.793081 808 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.792431 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.794622 807 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.794668 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.795269 801 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.795301 807 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.795296 772 leader_election.cc:310] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 6 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.795930 801 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Advancing to term 11 W0513 22:33:16.795969 772 raft_consensus.cc:2543] T 3c141683c43f4fb78c4489e7904b28e8 P 6fe2349260fd4c51a94a6bc60fa6043b: Unable to run election callback: Service unavailable: Thread pool token was shut down I0513 22:33:16.796762 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 9 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.796917 872 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.797652 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.797775 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.810600 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.811452 871 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.811439 944 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 9 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.812346 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.812402 939 raft_consensus.cc:1181] T a40718aa091247438d353d8c7739041e P fb91e39117a5489e85c5a494ee14bcc3 [term 1 FOLLOWER]: Refusing update from remote peer 6fe2349260fd4c51a94a6bc60fa6043b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I0513 22:33:16.812463 870 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 9. I0513 22:33:16.813637 806 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.813747 806 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.813750 803 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.814313 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.814435 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 9 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.814702 876 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 8 FOLLOWER]: Advancing to term 9 I0513 22:33:16.815557 997 consensus_queue.cc:984] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.003s I0513 22:33:16.816027 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.815909 807 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.817028 938 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 10 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.817229 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.817700 807 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 10 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.818166 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.820546 871 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 9. I0513 22:33:16.820538 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.821693 869 raft_consensus.cc:2321] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 10 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.821633 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.822388 1057 ts_tablet_manager.cc:1390] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 4.1 I0513 22:33:16.822466 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.822540 1057 log.cc:990] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/wals/01c2af1f12034771ba026a1fac9203df I0513 22:33:16.822912 801 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.823925 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.824204 804 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.824491 876 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 9. I0513 22:33:16.824662 805 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.824874 808 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.824937 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.824627 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.825214 944 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.824767 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.825522 1057 ts_tablet_manager.cc:1411] T 01c2af1f12034771ba026a1fac9203df P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting consensus metadata I0513 22:33:16.825666 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.826833 869 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.828649 1061 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Leader election decision vote started in defunct term 8: won I0513 22:33:16.828691 997 raft_consensus.cc:2603] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Leader election decision vote started in defunct term 8: won I0513 22:33:16.828799 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.829845 938 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.830549 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.831190 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.831781 801 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 11 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.832509 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.833185 939 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.834566 869 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.834547 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 8 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.834915 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.835811 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.835814 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.835899 938 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 11 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.836279 803 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.837353 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.838692 869 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 10. I0513 22:33:16.839109 872 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 9. I0513 22:33:16.839385 870 raft_consensus.cc:2321] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 10 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.839974 808 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 10 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.839937 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.840354 805 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.840736 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.840867 809 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.841225 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.841634 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.841789 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.842247 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.842336 803 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Advancing to term 12 I0513 22:33:16.842314 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.842645 944 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 10 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.843447 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.843850 939 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 10 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.843890 871 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Advancing to term 11 I0513 22:33:16.843569 806 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.844523 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.845379 997 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Leader election decision vote started in defunct term 8: won I0513 22:33:16.845542 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.845863 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.846721 799 tablet_service.cc:1057] Processing DeleteTablet for tablet 9fa827071e4246508d2690d2c1fbef4f with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.846719 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.847571 806 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 10 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.847971 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.848325 800 raft_consensus.cc:472] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.848431 800 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.850214 803 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.850473 1029 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Leader election decision vote started in defunct term 9: won I0513 22:33:16.851166 939 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.846901 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.851760 1057 tablet_replica.cc:264] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: stopping tablet replica I0513 22:33:16.851836 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.852015 809 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.852349 805 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.852442 805 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.852687 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.853106 1064 raft_consensus.cc:2588] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Leader election lost for term 7. Reason: could not achieve majority I0513 22:33:16.853098 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.853562 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 12 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.853708 809 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.854115 871 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 11. I0513 22:33:16.854519 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.854563 1029 raft_consensus.cc:2588] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority I0513 22:33:16.854799 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.854382 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.855490 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.855594 939 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 10 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 10 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.855489 800 raft_consensus.cc:494] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.855794 940 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 11 FOLLOWER]: Advancing to term 12 I0513 22:33:16.855850 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.855882 1059 raft_consensus.cc:2603] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Leader election decision vote started in defunct term 9: won I0513 22:33:16.856462 1029 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Leader election decision vote started in defunct term 9: won I0513 22:33:16.857023 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.857239 808 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.856719 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 10 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.857266 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.857767 871 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Advancing to term 10 I0513 22:33:16.858021 800 leader_election.cc:296] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 11 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.858247 803 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 12 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.859015 809 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.859073 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.859364 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.859884 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.860067 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.860412 803 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.859683 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.860638 1057 raft_consensus.cc:2143] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Raft consensus shutting down. I0513 22:33:16.860776 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.860929 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.861193 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.861332 806 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.861330 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.861594 806 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.862112 939 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.862316 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 9 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.862792 1029 raft_consensus.cc:2556] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Received election callback for term 10 while not running: Illegal state: RaftConsensus is not running: State = Stopping I0513 22:33:16.862828 804 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.863245 1057 raft_consensus.cc:2170] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Raft consensus is shut down! I0513 22:33:16.863245 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.863335 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.863417 802 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.863371 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.863436 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 12 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.863304 804 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Advancing to term 13 I0513 22:33:16.863735 876 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 11 FOLLOWER]: Advancing to term 12 I0513 22:33:16.864007 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.864552 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.865056 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 10 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.867544 809 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.868013 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.868592 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.867574 871 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 10 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 10. I0513 22:33:16.868815 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.869341 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.870177 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 10 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.870178 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.870681 940 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 12 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.870961 876 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 12. @ 0x7fa5784a2383 kudu::master::CatalogManager::ProcessTabletReport() at ??:0 I0513 22:33:16.873931 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.871389 805 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.874819 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.875697 1057 ts_tablet_manager.cc:1377] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting tablet data with delete state TABLET_DATA_DELETED I0513 22:33:16.876024 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.876045 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.875535 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.876397 809 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.876999 870 raft_consensus.cc:2886] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.878109 805 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 11 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.878901 806 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.880158 870 raft_consensus.cc:2346] T 9fa827071e4246508d2690d2c1fbef4f P 75a5cb20b56443639cfe49c47274d76b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 11. I0513 22:33:16.880163 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.880519 772 leader_election.cc:310] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.880657 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.880790 801 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.881572 1059 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Leader election decision vote started in defunct term 9: won I0513 22:33:16.881575 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.881697 1029 raft_consensus.cc:2588] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Leader election lost for term 10. Reason: could not achieve majority I0513 22:33:16.881731 804 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.881934 1064 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Leader election decision vote started in defunct term 11: won I0513 22:33:16.882376 1061 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Leader election decision vote started in defunct term 12: won I0513 22:33:16.881968 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.882756 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.882889 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.882994 804 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 13 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.883191 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.883251 997 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Leader election decision vote started in defunct term 10: won I0513 22:33:16.883399 939 raft_consensus.cc:2293] T 9fa827071e4246508d2690d2c1fbef4f P fb91e39117a5489e85c5a494ee14bcc3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 11 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.883744 806 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 11 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.883965 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.884106 871 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.884198 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.884920 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.885233 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.885807 944 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.885812 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.885587 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.886746 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.887318 804 raft_consensus.cc:472] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.887313 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.887322 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" W0513 22:33:16.887761 772 raft_consensus.cc:2543] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: Unable to run election callback: Service unavailable: Thread pool token was shut down I0513 22:33:16.888392 804 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Advancing to term 12 I0513 22:33:16.888801 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.889032 876 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 9 FOLLOWER]: Advancing to term 11 I0513 22:33:16.889989 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 @ 0x7fa5785e389b kudu::master::MasterServiceImpl::TSHeartbeat() at ??:0 I0513 22:33:16.890605 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 13 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.890758 938 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 12 FOLLOWER]: Advancing to term 13 I0513 22:33:16.891695 800 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.892367 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.892791 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.893316 809 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.893394 809 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Advancing to term 14 I0513 22:33:16.893335 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.894939 944 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 11 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.899282 803 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.899369 803 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 11 FOLLOWER]: Advancing to term 12 I0513 22:33:16.900182 870 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 13 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.900357 870 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 12 FOLLOWER]: Advancing to term 13 I0513 22:33:16.900640 938 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 13 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.900748 1057 ts_tablet_manager.cc:1390] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0 I0513 22:33:16.901031 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 11 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.901085 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.901199 940 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 10 FOLLOWER]: Advancing to term 11 I0513 22:33:16.901800 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.903764 804 raft_consensus.cc:494] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } @ 0x7fa5724dd44a std::_Function_handler<>::_M_invoke() at ??:0 I0513 22:33:16.904420 1057 log.cc:990] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/wals/9fa827071e4246508d2690d2c1fbef4f I0513 22:33:16.904664 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 12 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.904754 1057 ts_tablet_manager.cc:1411] T 9fa827071e4246508d2690d2c1fbef4f P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting consensus metadata I0513 22:33:16.904738 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 12 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.904889 938 raft_consensus.cc:2886] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 11 FOLLOWER]: Advancing to term 12 I0513 22:33:16.906630 938 raft_consensus.cc:2293] T e1484634c50f45f28345fdc882a61cf5 P fb91e39117a5489e85c5a494ee14bcc3 [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 12 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.906858 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.907198 871 raft_consensus.cc:2346] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 11. I0513 22:33:16.907150 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.907490 806 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.907500 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.907523 809 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.907788 997 raft_consensus.cc:2603] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Leader election decision vote started in defunct term 11: won I0513 22:33:16.908042 809 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 14 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.908098 799 tablet_service.cc:1057] Processing DeleteTablet for tablet e1484634c50f45f28345fdc882a61cf5 with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.908187 801 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.908305 1057 tablet_replica.cc:264] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: stopping tablet replica I0513 22:33:16.908324 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.908443 1057 raft_consensus.cc:2143] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Raft consensus shutting down. I0513 22:33:16.908547 1057 raft_consensus.cc:2170] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Raft consensus is shut down! I0513 22:33:16.908843 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 14 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.908974 938 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 13 FOLLOWER]: Advancing to term 14 I0513 22:33:16.909081 1057 ts_tablet_manager.cc:1377] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting tablet data with delete state TABLET_DATA_DELETED I0513 22:33:16.909076 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 14 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.909623 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.909665 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.909814 801 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.909974 870 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 13 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 13. I0513 22:33:16.910037 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.910296 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.910447 808 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 14 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.910575 808 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 14 FOLLOWER]: Advancing to term 15 I0513 22:33:16.910997 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.911394 872 raft_consensus.cc:2321] T e1484634c50f45f28345fdc882a61cf5 P 75a5cb20b56443639cfe49c47274d76b [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 12 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.911474 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.911613 800 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.911672 938 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 14 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.911877 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.911836 803 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.912317 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.912621 803 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 12 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.912603 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.912780 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 13 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.913041 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 12 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.913503 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.914294 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.914434 804 leader_election.cc:296] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 12 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.914607 871 raft_consensus.cc:2321] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 14 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.914965 940 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 11 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. @ 0x7fa56da239dc std::function<>::operator()() at ??:0 I0513 22:33:16.916998 808 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.917452 938 raft_consensus.cc:2321] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 11 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 12 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.917659 772 leader_election.cc:310] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 12 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.917812 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.917862 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.917994 803 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.918907 800 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.918853 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.918906 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.918709 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 15 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.919309 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.919438 809 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.919442 938 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 14 FOLLOWER]: Advancing to term 15 I0513 22:33:16.919745 809 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 12 FOLLOWER]: Advancing to term 13 I0513 22:33:16.920611 876 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 11 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 11. I0513 22:33:16.920884 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.921434 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.921519 997 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Leader election decision vote started in defunct term 13: won I0513 22:33:16.921587 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.922103 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 W0513 22:33:16.922188 772 raft_consensus.cc:2543] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: Unable to run election callback: Service unavailable: Thread pool token was shut down I0513 22:33:16.921988 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.922703 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.922711 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 14 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.923110 997 raft_consensus.cc:2588] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Leader election lost for term 14. Reason: could not achieve majority I0513 22:33:16.923296 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.923494 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 12 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.923687 809 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.923904 808 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 15 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.924175 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 15 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.924368 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.924930 800 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.924911 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.924911 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.925102 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.925141 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.925227 806 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.925232 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.924871 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.925580 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.924655 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 13 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.926069 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.926124 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.926337 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.926632 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 11 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.926937 809 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 13 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.927062 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.927155 876 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 11 FOLLOWER]: Advancing to term 12 I0513 22:33:16.927695 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.927914 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.927839 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 13 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.928019 804 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.928241 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.928263 805 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.928545 871 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 13 FOLLOWER]: Advancing to term 15 I0513 22:33:16.928573 997 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Leader election decision vote started in defunct term 11: won I0513 22:33:16.928942 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.928921 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.929054 804 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Advancing to term 16 I0513 22:33:16.929008 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.929509 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.930917 940 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 11 FOLLOWER]: Advancing to term 13 I0513 22:33:16.931169 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.932056 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.932503 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.933076 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.933357 938 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 15 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.933802 809 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.933882 800 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.934031 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.934062 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.934064 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.934322 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.934515 807 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.934965 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.935163 807 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.935246 803 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.935166 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.936188 1057 ts_tablet_manager.cc:1390] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0 I0513 22:33:16.936240 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.936280 809 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 13 FOLLOWER]: Advancing to term 14 I0513 22:33:16.935328 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.936905 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.936933 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.937081 802 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.937338 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.937678 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.938155 805 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.938632 876 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 12 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 12. I0513 22:33:16.938814 872 raft_consensus.cc:2321] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 12 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 13 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.938987 871 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 15 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 15. I0513 22:33:16.939050 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 12 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.939721 809 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 14 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.940289 997 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 14 FOLLOWER]: Leader election decision vote started in defunct term 12: won I0513 22:33:16.940440 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 15 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.941130 809 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 14 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.941521 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.941540 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.942106 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.942342 1057 log.cc:990] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/wals/e1484634c50f45f28345fdc882a61cf5 I0513 22:33:16.942476 876 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 14 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.942610 876 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 12 FOLLOWER]: Advancing to term 14 I0513 22:33:16.943405 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.943389 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 14 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.943614 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.944165 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.944257 802 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.944607 808 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.944799 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.944672 940 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 13 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.945384 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.945688 773 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 13 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.945734 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.946143 938 raft_consensus.cc:2321] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 13 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 14 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.945274 804 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.946238 802 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 14 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.946316 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.946347 802 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 14 FOLLOWER]: Advancing to term 15 I0513 22:33:16.946506 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.947440 804 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 16 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) @ 0x7fa56da2254b kudu::rpc::GeneratedServiceIf::Handle() at ??:0 I0513 22:33:16.948053 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.948127 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.948384 1057 ts_tablet_manager.cc:1411] T e1484634c50f45f28345fdc882a61cf5 P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting consensus metadata I0513 22:33:16.948586 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.948773 997 raft_consensus.cc:2603] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 FOLLOWER]: Leader election decision vote started in defunct term 15: won I0513 22:33:16.948940 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 16 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.949208 802 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.949434 1061 raft_consensus.cc:2588] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Leader election lost for term 13. Reason: could not achieve majority I0513 22:33:16.949852 801 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.950394 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 15 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.950546 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.950644 802 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 15 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.950775 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.951005 871 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 15 FOLLOWER]: Advancing to term 16 I0513 22:33:16.951241 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.951366 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.951391 940 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 15 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.951483 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.951694 876 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 14 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 14. I0513 22:33:16.951923 806 raft_consensus.cc:472] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.952131 940 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 13 FOLLOWER]: Advancing to term 15 I0513 22:33:16.952141 872 raft_consensus.cc:2321] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 14 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 15 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.952188 806 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 FOLLOWER]: Advancing to term 17 I0513 22:33:16.952276 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 14 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.952451 805 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.952646 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.952644 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.952807 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.953028 1061 raft_consensus.cc:2603] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Leader election decision vote started in defunct term 14: won I0513 22:33:16.953658 799 tablet_service.cc:1057] Processing DeleteTablet for tablet 7e57e922f3a84a2e99667b5eacf7c18e with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.953889 1057 tablet_replica.cc:264] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: stopping tablet replica I0513 22:33:16.953887 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.954080 804 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.953656 938 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 16 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.954267 938 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 15 FOLLOWER]: Advancing to term 16 I0513 22:33:16.955720 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.956054 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.956249 804 raft_consensus.cc:472] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Starting forced leader election (received explicit request) I0513 22:33:16.956255 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.956254 806 raft_consensus.cc:494] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 17 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.956452 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.957093 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.957263 805 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.956746 871 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 16. I0513 22:33:16.957856 806 leader_election.cc:296] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 17 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.957890 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.958251 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: I0513 22:33:16.958329 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.958390 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.958575 1057 raft_consensus.cc:2143] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 17 FOLLOWER]: Raft consensus shutting down. I0513 22:33:16.958628 944 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 17 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" I0513 22:33:16.958703 1061 raft_consensus.cc:2556] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 17 FOLLOWER]: Received election callback for term 16 while not running: Illegal state: RaftConsensus is not running: State = Stopping I0513 22:33:16.958809 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.958925 806 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.959100 871 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 17 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.959157 804 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 15 FOLLOWER]: Advancing to term 16 I0513 22:33:16.959480 871 raft_consensus.cc:2886] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 16 FOLLOWER]: Advancing to term 17 I0513 22:33:16.959941 1057 raft_consensus.cc:2170] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [term 17 FOLLOWER]: Raft consensus is shut down! I0513 22:33:16.960701 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.961225 805 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.961441 809 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.961473 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.961975 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.962168 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.960877 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.963673 804 raft_consensus.cc:494] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.964469 872 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 16 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "75a5cb20b56443639cfe49c47274d76b" I0513 22:33:16.964475 939 tablet_service.cc:1286] Received RequestConsensusVote() RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" candidate_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" candidate_term: 16 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "fb91e39117a5489e85c5a494ee14bcc3" @ 0x7fa56da26048 kudu::rpc::ServicePool::RunThread() at ??:0 I0513 22:33:16.967983 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.968189 938 raft_consensus.cc:2293] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 16 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.968484 1057 ts_tablet_manager.cc:1377] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting tablet data with delete state TABLET_DATA_DELETED I0513 22:33:16.968513 944 raft_consensus.cc:2321] T 7e57e922f3a84a2e99667b5eacf7c18e P fb91e39117a5489e85c5a494ee14bcc3 [term 16 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 17 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.968938 872 raft_consensus.cc:2886] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 14 FOLLOWER]: Advancing to term 16 I0513 22:33:16.969143 940 raft_consensus.cc:2293] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 15 because replica has last-logged OpId of term: 1 index: 1, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0. I0513 22:33:16.969146 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.969419 773 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 15 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b; no voters: 75a5cb20b56443639cfe49c47274d76b, fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.970058 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.970057 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.970305 802 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.970345 806 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.970350 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.972003 939 raft_consensus.cc:2321] T 9ce2cd8e3814425b86d4ef0b5309b05c P fb91e39117a5489e85c5a494ee14bcc3 [term 15 FOLLOWER]: Leader election vote request: Denying vote to candidate 6fe2349260fd4c51a94a6bc60fa6043b for term 16 because replica is already servicing an update from a current leader or another vote. I0513 22:33:16.971982 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.972350 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.972628 1061 raft_consensus.cc:2588] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 FOLLOWER]: Leader election lost for term 15. Reason: could not achieve majority I0513 22:33:16.973177 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.973575 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.973639 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.973474 804 leader_election.cc:296] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 16 election: Requested vote from peers 75a5cb20b56443639cfe49c47274d76b (127.0.116.130:35393), fb91e39117a5489e85c5a494ee14bcc3 (127.0.116.131:36837) I0513 22:33:16.973891 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.974102 872 raft_consensus.cc:2346] T 9ce2cd8e3814425b86d4ef0b5309b05c P 75a5cb20b56443639cfe49c47274d76b [term 16 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 16. I0513 22:33:16.974308 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.974413 801 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.974383 772 leader_election.cc:310] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 16 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.974617 1061 raft_consensus.cc:2640] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 FOLLOWER]: Leader election won for term 16 I0513 22:33:16.974699 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.974730 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.975169 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.975814 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.975852 1061 raft_consensus.cc:667] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 LEADER]: Becoming Leader. State: Replica: 6fe2349260fd4c51a94a6bc60fa6043b, State: Running, Role: LEADER I0513 22:33:16.976019 806 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.976063 1061 consensus_queue.cc:206] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 16, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" member_type: VOTER last_known_addr { host: "127.0.116.129" port: 34317 } } peers { permanent_uuid: "75a5cb20b56443639cfe49c47274d76b" member_type: VOTER last_known_addr { host: "127.0.116.130" port: 35393 } } peers { permanent_uuid: "fb91e39117a5489e85c5a494ee14bcc3" member_type: VOTER last_known_addr { host: "127.0.116.131" port: 36837 } } I0513 22:33:16.976567 871 raft_consensus.cc:2346] T 7e57e922f3a84a2e99667b5eacf7c18e P 75a5cb20b56443639cfe49c47274d76b [term 17 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fe2349260fd4c51a94a6bc60fa6043b in term 17. I0513 22:33:16.977272 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.977277 772 leader_election.cc:310] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b [CANDIDATE]: Term 17 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 6fe2349260fd4c51a94a6bc60fa6043b, 75a5cb20b56443639cfe49c47274d76b; no voters: fb91e39117a5489e85c5a494ee14bcc3 I0513 22:33:16.977627 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.977478 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 W0513 22:33:16.977906 772 raft_consensus.cc:2543] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: Unable to run election callback: Service unavailable: Thread pool token was shut down I0513 22:33:16.977926 801 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.977929 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.978045 808 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.978547 806 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.978819 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.978826 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.979393 802 raft_consensus.cc:455] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.979387 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.979650 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.979920 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.980412 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.980746 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.981068 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.981289 802 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.981395 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.981508 1057 ts_tablet_manager.cc:1390] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 0.0 I0513 22:33:16.981640 1057 log.cc:990] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/wals/7e57e922f3a84a2e99667b5eacf7c18e I0513 22:33:16.981827 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.981935 1057 ts_tablet_manager.cc:1411] T 7e57e922f3a84a2e99667b5eacf7c18e P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting consensus metadata @ 0x7fa56da2b573 boost::_bi::bind_t<>::operator()() at ??:0 I0513 22:33:16.982121 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.982471 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.982851 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.983065 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.983245 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.983361 802 raft_consensus.cc:455] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.983650 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.984050 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.984484 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.984890 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.984987 802 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.985273 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.985383 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.985711 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 @ 0x7fa56d9861ac boost::function0<>::operator()() at ??:0 I0513 22:33:16.986039 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.986443 799 tablet_service.cc:1057] Processing DeleteTablet for tablet 9ce2cd8e3814425b86d4ef0b5309b05c with delete_type TABLET_DATA_DELETED from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.986742 1057 tablet_replica.cc:264] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: stopping tablet replica I0513 22:33:16.986800 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.986922 802 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.986937 1057 raft_consensus.cc:2143] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 LEADER]: Raft consensus shutting down. I0513 22:33:16.987097 1057 pending_rounds.cc:64] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: Trying to abort 1 pending transactions. I0513 22:33:16.987134 1057 pending_rounds.cc:70] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: Aborting transaction as it isn't in flight: id { term: 16 index: 1 } timestamp: 6380694720415891456 op_type: NO_OP noop_request { } I0513 22:33:16.987263 1057 raft_consensus.cc:2723] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 LEADER]: NO_OP replication failed: Aborted: Transaction aborted I0513 22:33:16.987350 1057 raft_consensus.cc:2170] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b [term 16 FOLLOWER]: Raft consensus is shut down! I0513 22:33:16.987723 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.988025 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.988361 1057 ts_tablet_manager.cc:1377] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting tablet data with delete state TABLET_DATA_DELETED @ 0x7fa565245ccb kudu::Thread::SuperviseThread() at ??:0 I0513 22:33:16.990478 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.990581 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.991525 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.991645 802 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.991916 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.992568 1057 ts_tablet_manager.cc:1390] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 16.1 I0513 22:33:16.992739 1057 log.cc:990] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting WAL directory at /tmp/dist-test-taskxM53nH/test-tmp/delete_tablet-itest.0.DeleteTabletITest.TestLeaderElectionDuringDeleteTablet.1557786794576917-466/minicluster-data/ts-0-root/wals/9ce2cd8e3814425b86d4ef0b5309b05c I0513 22:33:16.992776 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.993221 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 @ 0x7fa56892c184 start_thread at ??:0 I0513 22:33:16.994458 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.994917 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "a40718aa091247438d353d8c7739041e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.995038 808 raft_consensus.cc:455] T a40718aa091247438d353d8c7739041e P 6fe2349260fd4c51a94a6bc60fa6043b [term 1 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.995074 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9ce2cd8e3814425b86d4ef0b5309b05c" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.995149 1057 ts_tablet_manager.cc:1411] T 9ce2cd8e3814425b86d4ef0b5309b05c P 6fe2349260fd4c51a94a6bc60fa6043b: Deleting consensus metadata I0513 22:33:16.995641 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "7e57e922f3a84a2e99667b5eacf7c18e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.995908 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "3c141683c43f4fb78c4489e7904b28e8" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.996019 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "e1484634c50f45f28345fdc882a61cf5" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.996543 802 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "73f1725c5aa04760a9d5e43d09f66e9e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.996541 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "8002a0fb94514f77a8601f629424436e" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.996655 808 raft_consensus.cc:455] T 8002a0fb94514f77a8601f629424436e P 6fe2349260fd4c51a94a6bc60fa6043b [term 3 LEADER]: Not starting forced leader election -- already a leader I0513 22:33:16.997534 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "c1cbbc2b2f22460aa0617e7a5c823a57" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.997925 804 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "01c2af1f12034771ba026a1fac9203df" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 I0513 22:33:16.998034 808 tablet_service.cc:1403] Received Run Leader Election RPC: tablet_id: "9fa827071e4246508d2690d2c1fbef4f" dest_uuid: "6fe2349260fd4c51a94a6bc60fa6043b" from {username='slave'} at 127.0.0.1:57614 @ 0x7fa560dc1ffd clone at ??:0