I1031 23:03:52.451198 10759 tablet_service.cc:650] Processing CreateTablet for tablet 68aa6071fa594e79958cddb8489a1f0c (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171025 <= VALUES < 20171026 I1031 23:03:52.451217 10761 tablet_service.cc:650] Processing CreateTablet for tablet 5110f0095bd94f67abf0aad906332efe (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171028 <= VALUES < 20171029 I1031 23:03:52.451215 10772 tablet_service.cc:650] Processing CreateTablet for tablet f49a4f915ac048b096fcb5f4878f436d (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171029 <= VALUES < 20171030 I1031 23:03:52.451198 10756 tablet_service.cc:650] Processing CreateTablet for tablet c7eb286102a648969b5cf2a93710a72c (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171027 <= VALUES < 20171028 I1031 23:03:52.451225 10764 tablet_service.cc:650] Processing CreateTablet for tablet 318722d72b774164aba6657098b58c90 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171030 <= VALUES < 20171031 I1031 23:03:52.451264 10773 tablet_service.cc:650] Processing CreateTablet for tablet 67cc714357a24166a1354f3d7276848f (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171031 <= VALUES < 20171101 I1031 23:03:52.451278 10760 tablet_service.cc:650] Processing CreateTablet for tablet 6f911a94e20d47aa9efaf9ffe3914c3b (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171101 <= VALUES < 20171102 I1031 23:03:52.451290 10768 tablet_service.cc:650] Processing CreateTablet for tablet d528d43805c44cc398cfba41658e36f0 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171023 <= VALUES < 20171024 I1031 23:03:52.451298 10767 tablet_service.cc:650] Processing CreateTablet for tablet f05a7aa04e784d02a30ce666139a7263 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171024 <= VALUES < 20171025 I1031 23:03:52.451355 10769 tablet_service.cc:650] Processing CreateTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171030 <= VALUES < 20171031 I1031 23:03:52.451359 10758 tablet_service.cc:650] Processing CreateTablet for tablet bd6a9e583cad413ca558178148ac7f85 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171026 <= VALUES < 20171027 I1031 23:03:52.451375 10765 tablet_service.cc:650] Processing CreateTablet for tablet 69b8c3e3568c4c19b638bb66e46a95a1 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171101 <= VALUES < 20171102 I1031 23:03:52.451400 10762 tablet_service.cc:650] Processing CreateTablet for tablet feeb15ca2b1f490296ccd8a8edacf49f (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171029 <= VALUES < 20171030 I1031 23:03:52.451411 10770 tablet_service.cc:650] Processing CreateTablet for tablet 6ca74226a6af40e2a78046fc3591c038 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 2, RANGE (date) PARTITION 20171024 <= VALUES < 20171025 I1031 23:03:52.451436 10763 tablet_service.cc:650] Processing CreateTablet for tablet 269d7d8c1d21484ab7f554a011190a84 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 2, RANGE (date) PARTITION 20171025 <= VALUES < 20171026 I1031 23:03:52.451448 10771 tablet_service.cc:650] Processing CreateTablet for tablet 9967925d5dbd49eaadc0a1a58e7e2089 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 2, RANGE (date) PARTITION 20171028 <= VALUES < 20171029 I1031 23:03:52.451477 10766 tablet_service.cc:650] Processing CreateTablet for tablet 8182189275034354a3c350318114c7e4 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 2, RANGE (date) PARTITION 20171030 <= VALUES < 20171031 I1031 23:03:52.451517 10755 tablet_service.cc:650] Processing CreateTablet for tablet 3125380200594db7a7b78a5539b9dfad (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 2, RANGE (date) PARTITION 20171101 <= VALUES < 20171102 I1031 23:03:52.451548 10754 tablet_service.cc:650] Processing CreateTablet for tablet 1c64d9f8e8c740c8ae73f0d7357fcbfd (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 2, RANGE (date) PARTITION 20171029 <= VALUES < 20171030 I1031 23:03:52.451663 10757 tablet_service.cc:650] Processing CreateTablet for tablet fcde65c4e4cf4df29b9ef9884ce292b2 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 2, RANGE (date) PARTITION 20171031 <= VALUES < 20171101 I1031 23:03:52.589109 10769 ts_tablet_manager.cc:862] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.589510 39728 ts_tablet_manager.cc:723] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.589519 10769 tablet_service.cc:650] Processing CreateTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171023 <= VALUES < 20171024 I1031 23:03:52.589890 39728 tablet_bootstrap.cc:382] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.590054 39728 tablet_bootstrap.cc:541] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.590183 39728 tablet_bootstrap.cc:484] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.590978 39728 tablet_bootstrap.cc:382] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.591008 39728 ts_tablet_manager.cc:728] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s I1031 23:03:52.591698 39728 raft_consensus.cc:288] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:03:52.591727 39728 raft_consensus.cc:319] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.591743 39728 raft_consensus.cc:548] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.591776 39728 consensus_queue.cc:176] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.614400 10763 ts_tablet_manager.cc:862] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.614871 39733 ts_tablet_manager.cc:723] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.614938 10763 tablet_service.cc:650] Processing CreateTablet for tablet e53178c96c3c4ad2b3068e9674d1259d (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171025 <= VALUES < 20171026 I1031 23:03:52.615263 39733 tablet_bootstrap.cc:382] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.615398 39733 tablet_bootstrap.cc:541] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.615527 39733 tablet_bootstrap.cc:484] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.633394 39733 tablet_bootstrap.cc:382] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.633447 39733 ts_tablet_manager.cc:728] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.019s user 0.001s sys 0.000s I1031 23:03:52.634541 39733 raft_consensus.cc:288] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:03:52.634569 39733 raft_consensus.cc:319] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.634583 39733 raft_consensus.cc:548] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.634604 39733 consensus_queue.cc:176] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.639906 10757 ts_tablet_manager.cc:862] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.640473 39738 ts_tablet_manager.cc:723] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.640542 10757 tablet_service.cc:650] Processing CreateTablet for tablet 388697c0c21a4b158a92f2b292b8012f (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171028 <= VALUES < 20171029 I1031 23:03:52.641031 39738 tablet_bootstrap.cc:382] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.641230 39738 tablet_bootstrap.cc:541] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.641420 39738 tablet_bootstrap.cc:484] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.642702 39738 tablet_bootstrap.cc:382] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.642742 39738 ts_tablet_manager.cc:728] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I1031 23:03:52.643546 39738 raft_consensus.cc:288] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0f53a0d3ef7e44ebb0365c800752d5bd" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st01.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:03:52.643577 39738 raft_consensus.cc:319] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.643599 39738 raft_consensus.cc:548] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.643627 39738 consensus_queue.cc:176] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.665231 10756 ts_tablet_manager.cc:862] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.665997 39743 ts_tablet_manager.cc:723] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.666102 10756 tablet_service.cc:650] Processing CreateTablet for tablet 090accc17fae4e60b7bddb6dd0bf2ef8 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171030 <= VALUES < 20171031 I1031 23:03:52.666453 39743 tablet_bootstrap.cc:382] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.666615 39743 tablet_bootstrap.cc:541] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.666772 39743 tablet_bootstrap.cc:484] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.668025 39743 tablet_bootstrap.cc:382] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.668052 39743 ts_tablet_manager.cc:728] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s I1031 23:03:52.668741 39743 raft_consensus.cc:288] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:03:52.668764 39743 raft_consensus.cc:319] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.668776 39743 raft_consensus.cc:548] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.668790 39743 consensus_queue.cc:176] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.690452 10770 ts_tablet_manager.cc:862] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.690775 39748 ts_tablet_manager.cc:723] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.690874 10770 tablet_service.cc:650] Processing CreateTablet for tablet 865b2d5788f44598b169ca6415314610 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171031 <= VALUES < 20171101 I1031 23:03:52.691248 39748 tablet_bootstrap.cc:382] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.691402 39748 tablet_bootstrap.cc:541] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.691551 39748 tablet_bootstrap.cc:484] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.692754 39748 tablet_bootstrap.cc:382] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.692786 39748 ts_tablet_manager.cc:728] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s I1031 23:03:52.693647 39748 raft_consensus.cc:288] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:03:52.693673 39748 raft_consensus.cc:319] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.693684 39748 raft_consensus.cc:548] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.693708 39748 consensus_queue.cc:176] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.715920 10767 ts_tablet_manager.cc:862] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.716204 39753 ts_tablet_manager.cc:723] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.716294 10767 tablet_service.cc:650] Processing CreateTablet for tablet 1d674793183b4a3fb8f88dc864e9059a (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171024 <= VALUES < 20171025 I1031 23:03:52.716699 39753 tablet_bootstrap.cc:382] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.716897 39753 tablet_bootstrap.cc:541] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.717036 39753 tablet_bootstrap.cc:484] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.718040 39753 tablet_bootstrap.cc:382] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.718070 39753 ts_tablet_manager.cc:728] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s I1031 23:03:52.719094 39753 raft_consensus.cc:288] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } I1031 23:03:52.719118 39753 raft_consensus.cc:319] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.719131 39753 raft_consensus.cc:548] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.719149 39753 consensus_queue.cc:176] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.730764 10777 raft_consensus.cc:2162] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.582s I1031 23:03:52.730886 10777 raft_consensus.cc:1747] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fc477287c154136abc1e2f3c82fb7f9 in term 0. I1031 23:03:52.734125 10777 raft_consensus.cc:2206] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:03:52.734175 10777 raft_consensus.cc:2162] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.596s I1031 23:03:52.734948 10777 raft_consensus.cc:1747] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fc477287c154136abc1e2f3c82fb7f9 in term 1. I1031 23:03:52.741122 10764 ts_tablet_manager.cc:862] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.741399 39758 ts_tablet_manager.cc:723] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.741482 10764 tablet_service.cc:650] Processing CreateTablet for tablet 812b248175004ae8992ff2660fc9d520 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171025 <= VALUES < 20171026 I1031 23:03:52.741804 39758 tablet_bootstrap.cc:382] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.742000 39758 tablet_bootstrap.cc:541] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.742121 39758 tablet_bootstrap.cc:484] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.743332 39758 tablet_bootstrap.cc:382] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.743367 39758 ts_tablet_manager.cc:728] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s I1031 23:03:52.743893 39758 raft_consensus.cc:288] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:03:52.743917 39758 raft_consensus.cc:319] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.743929 39758 raft_consensus.cc:548] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.744012 39758 consensus_queue.cc:176] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.774662 10762 ts_tablet_manager.cc:862] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.775009 39763 ts_tablet_manager.cc:723] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.775080 10762 tablet_service.cc:650] Processing CreateTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171027 <= VALUES < 20171028 I1031 23:03:52.775485 39763 tablet_bootstrap.cc:382] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.775655 39763 tablet_bootstrap.cc:541] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.775802 39763 tablet_bootstrap.cc:484] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.776644 39763 tablet_bootstrap.cc:382] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.776676 39763 ts_tablet_manager.cc:728] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s I1031 23:03:52.777155 39763 raft_consensus.cc:288] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:03:52.777179 39763 raft_consensus.cc:319] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.777194 39763 raft_consensus.cc:548] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.777215 39763 consensus_queue.cc:176] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.800243 10759 ts_tablet_manager.cc:862] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.800545 39768 ts_tablet_manager.cc:723] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:03:52.800609 10759 tablet_service.cc:650] Processing CreateTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171031 <= VALUES < 20171101 I1031 23:03:52.801038 39768 tablet_bootstrap.cc:382] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:03:52.801921 39768 tablet_bootstrap.cc:541] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:03:52.802064 39768 tablet_bootstrap.cc:484] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:03:52.802664 39768 tablet_bootstrap.cc:382] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:03:52.802691 39768 ts_tablet_manager.cc:728] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I1031 23:03:52.803131 39768 raft_consensus.cc:288] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:03:52.803160 39768 raft_consensus.cc:319] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:03:52.803170 39768 raft_consensus.cc:548] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:03:52.803186 39768 consensus_queue.cc:176] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:03:52.825320 10754 ts_tablet_manager.cc:862] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.825408 10754 tablet_service.cc:650] Processing CreateTablet for tablet 648eb376255a4dd4bb5818262efcc813 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171101 <= VALUES < 20171102 I1031 23:03:52.844645 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:03:52.844683 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.932s I1031 23:03:52.844735 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:03:52.845420 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:03:52.845468 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:03:52.845679 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:03:52.845738 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Not found: Tablet not found I1031 23:03:52.845759 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:03:52.845841 39732 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.826s I1031 23:03:52.845865 39732 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:03:52.850719 10761 ts_tablet_manager.cc:862] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.875849 10765 ts_tablet_manager.cc:862] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.900943 10755 ts_tablet_manager.cc:862] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.926308 10766 ts_tablet_manager.cc:862] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.941737 39746 raft_consensus.cc:411] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:03:52.941768 39746 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.697s I1031 23:03:52.941812 39746 raft_consensus.cc:435] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:03:52.942502 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:03:52.942535 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 W1031 23:03:52.942850 10718 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:03:52.942910 10719 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3eb30bd1bc34b95902fa61724239553: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:03:52.942939 10719 leader_election.cc:243] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:03:52.942977 39747 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.127s I1031 23:03:52.943002 39747 raft_consensus.cc:1908] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:03:52.959833 10760 ts_tablet_manager.cc:862] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:52.960705 39751 raft_consensus.cc:411] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:03:52.960753 39751 raft_consensus.cc:2162] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.140s I1031 23:03:52.960830 39751 raft_consensus.cc:435] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:03:52.961665 39751 leader_election.cc:216] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:03:52.961695 39751 leader_election.cc:216] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:03:52.961906 10719 leader_election.cc:348] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer 193181be29284287b25df2ee92938f0b I1031 23:03:52.961926 10719 leader_election.cc:243] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:03:52.962019 39752 raft_consensus.cc:2162] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.978s I1031 23:03:52.962033 10720 leader_election.cc:348] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:03:52.962044 39752 raft_consensus.cc:1971] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:03:52.962082 39752 raft_consensus.cc:411] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:03:52.962100 39752 raft_consensus.cc:2162] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.102s I1031 23:03:52.962111 39752 raft_consensus.cc:2206] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:03:52.962774 39752 raft_consensus.cc:435] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:03:52.963279 39752 leader_election.cc:216] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 193181be29284287b25df2ee92938f0b I1031 23:03:52.963305 39752 leader_election.cc:216] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:03:52.963963 10720 leader_election.cc:348] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:03:52.963987 10720 leader_election.cc:243] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:03:52.964031 39752 raft_consensus.cc:2162] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.036s I1031 23:03:52.964046 39752 raft_consensus.cc:1971] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:03:52.964357 39752 raft_consensus.cc:518] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:03:52.964423 39752 consensus_queue.cc:158] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:03:52.964895 10719 leader_election.cc:348] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer 193181be29284287b25df2ee92938f0b I1031 23:03:52.985360 10777 raft_consensus.cc:1677] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 625f49b9554549e6a5c3a92b76e576cf in current term 1: Already voted for candidate 0fc477287c154136abc1e2f3c82fb7f9 in this term. I1031 23:03:53.025158 39771 raft_consensus.cc:411] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:03:53.025245 39771 raft_consensus.cc:2162] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.942s I1031 23:03:53.025362 39771 raft_consensus.cc:435] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:03:53.026217 39771 leader_election.cc:216] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:03:53.026278 39771 leader_election.cc:216] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 0fc477287c154136abc1e2f3c82fb7f9 W1031 23:03:53.026501 10720 leader_election.cc:278] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Not found: Tablet not found I1031 23:03:53.026556 10718 leader_election.cc:348] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:03:53.026581 10718 leader_election.cc:243] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:03:53.026648 39772 raft_consensus.cc:2162] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.841s I1031 23:03:53.026669 39772 raft_consensus.cc:1971] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:03:53.026682 39772 raft_consensus.cc:411] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:03:53.026697 39772 raft_consensus.cc:2162] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.185s I1031 23:03:53.026710 39772 raft_consensus.cc:2206] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:03:53.059195 39766 raft_consensus.cc:411] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:03:53.059269 39766 raft_consensus.cc:2162] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.614s I1031 23:03:53.059357 39766 raft_consensus.cc:435] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:03:53.059940 39766 leader_election.cc:216] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer daba39963d4346e1816089fd3a5442b5 I1031 23:03:53.059979 39766 leader_election.cc:216] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:03:53.060161 10717 leader_election.cc:348] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer daba39963d4346e1816089fd3a5442b5 I1031 23:03:53.060181 10717 leader_election.cc:243] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. W1031 23:03:53.060209 10720 leader_election.cc:278] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 044f1da9a27c46acb82b1386f829f4dc: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:03:53.060230 39767 raft_consensus.cc:2162] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.598s I1031 23:03:53.060250 39767 raft_consensus.cc:1971] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:03:53.060263 39767 raft_consensus.cc:411] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:03:53.060277 39767 raft_consensus.cc:2162] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.197s I1031 23:03:53.060289 39767 raft_consensus.cc:2206] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:03:53.072903 10777 raft_consensus.cc:2162] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.050s I1031 23:03:53.073047 10777 raft_consensus.cc:1747] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate daba39963d4346e1816089fd3a5442b5 in term 0. I1031 23:03:53.074448 10777 raft_consensus.cc:2206] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:03:53.074502 10777 raft_consensus.cc:2162] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.865s I1031 23:03:53.111121 39767 raft_consensus.cc:435] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:03:53.111142 10777 raft_consensus.cc:1747] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate daba39963d4346e1816089fd3a5442b5 in term 1. I1031 23:03:53.111264 39772 raft_consensus.cc:435] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:03:53.111804 39767 leader_election.cc:216] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer daba39963d4346e1816089fd3a5442b5 I1031 23:03:53.111834 39772 leader_election.cc:216] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:03:53.111836 39767 leader_election.cc:216] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:03:53.111866 39772 leader_election.cc:216] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 0fc477287c154136abc1e2f3c82fb7f9 W1031 23:03:53.112071 10720 leader_election.cc:278] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 044f1da9a27c46acb82b1386f829f4dc: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:03:53.112161 10720 leader_election.cc:278] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:03:53.112779 10717 leader_election.cc:348] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer daba39963d4346e1816089fd3a5442b5 I1031 23:03:53.112797 10717 leader_election.cc:243] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:03:53.112838 39767 raft_consensus.cc:2162] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.206s I1031 23:03:53.112854 39767 raft_consensus.cc:1971] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:03:53.113081 10718 leader_election.cc:348] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:03:53.113095 39767 raft_consensus.cc:518] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:03:53.113101 10718 leader_election.cc:243] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:03:53.113142 39772 raft_consensus.cc:2162] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.007s I1031 23:03:53.113154 39767 consensus_queue.cc:158] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:03:53.113162 39772 raft_consensus.cc:1971] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:03:53.113896 39772 raft_consensus.cc:518] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:03:53.113972 39772 consensus_queue.cc:158] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:03:53.131762 10777 raft_consensus.cc:2162] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.652s I1031 23:03:53.131821 10777 raft_consensus.cc:1747] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f53a0d3ef7e44ebb0365c800752d5bd in term 0. I1031 23:03:53.133095 10777 raft_consensus.cc:2206] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:03:53.133119 10777 raft_consensus.cc:2162] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.592s I1031 23:03:53.161265 10777 raft_consensus.cc:1747] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f53a0d3ef7e44ebb0365c800752d5bd in term 1. I1031 23:03:53.161347 10772 ts_tablet_manager.cc:862] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.172559 39736 raft_consensus.cc:411] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:03:53.172626 39736 raft_consensus.cc:2162] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.126s I1031 23:03:53.172694 39736 raft_consensus.cc:435] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:03:53.173311 39736 leader_election.cc:216] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:03:53.173341 39736 leader_election.cc:216] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer daba39963d4346e1816089fd3a5442b5 I1031 23:03:53.173642 10720 leader_election.cc:348] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:03:53.173665 10720 leader_election.cc:243] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:03:53.173682 10717 leader_election.cc:348] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer daba39963d4346e1816089fd3a5442b5 I1031 23:03:53.173710 39737 raft_consensus.cc:2162] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.240s I1031 23:03:53.173730 39737 raft_consensus.cc:1971] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:03:53.173743 39737 raft_consensus.cc:411] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:03:53.173758 39737 raft_consensus.cc:2162] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.905s I1031 23:03:53.173768 39737 raft_consensus.cc:2206] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:03:53.194146 10771 ts_tablet_manager.cc:862] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.194310 39737 raft_consensus.cc:435] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:03:53.195047 39737 leader_election.cc:216] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:03:53.195080 39737 leader_election.cc:216] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer daba39963d4346e1816089fd3a5442b5 I1031 23:03:53.195943 10720 leader_election.cc:348] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:03:53.195966 10720 leader_election.cc:243] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:03:53.195984 10717 leader_election.cc:348] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer daba39963d4346e1816089fd3a5442b5 I1031 23:03:53.196003 39737 raft_consensus.cc:2162] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.057s I1031 23:03:53.196019 39737 raft_consensus.cc:1971] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:03:53.196269 39737 raft_consensus.cc:518] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:03:53.196327 39737 consensus_queue.cc:158] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:03:53.235985 10768 ts_tablet_manager.cc:862] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.236649 10779 raft_consensus.cc:918] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 0fc477287c154136abc1e2f3c82fb7f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:03:53.262161 10773 ts_tablet_manager.cc:862] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.269979 10758 ts_tablet_manager.cc:862] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.287230 10756 ts_tablet_manager.cc:862] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.362799 10770 ts_tablet_manager.cc:862] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.388384 10762 ts_tablet_manager.cc:862] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.413440 10759 ts_tablet_manager.cc:862] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.438684 10769 ts_tablet_manager.cc:862] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.463966 10754 ts_tablet_manager.cc:862] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.464040 10754 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.CreateTablet from 10.108.187.10:43437 (request call id 30) took 1012ms. Request Metrics: {"fdatasync":1,"fdatasync_us":58824} I1031 23:03:53.465544 39752 consensus_queue.cc:695] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 193181be29284287b25df2ee92938f0b, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:03:53.466047 39752 consensus_queue.cc:695] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 23f962e4a1744381ad5fa0d2d8b10241, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:03:53.489105 10763 ts_tablet_manager.cc:862] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.489285 10763 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.CreateTablet from 10.108.187.10:43437 (request call id 22) took 1037ms. Request Metrics: {"fdatasync":1,"fdatasync_us":58620} I1031 23:03:53.514363 10767 ts_tablet_manager.cc:862] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.514418 10767 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.CreateTablet from 10.108.187.10:43437 (request call id 26) took 1062ms. Request Metrics: {"fdatasync":1,"fdatasync_us":49904} I1031 23:03:53.539567 10764 ts_tablet_manager.cc:862] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.539611 10764 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.CreateTablet from 10.108.187.10:43437 (request call id 27) took 1088ms. Request Metrics: {"fdatasync":1,"fdatasync_us":49901} I1031 23:03:53.564606 10757 ts_tablet_manager.cc:862] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:03:53.564656 10757 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.CreateTablet from 10.108.187.10:43437 (request call id 23) took 1113ms. Request Metrics: {"fdatasync":1,"fdatasync_us":58826} I1031 23:03:53.612984 10782 raft_consensus.cc:918] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer daba39963d4346e1816089fd3a5442b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:03:53.614928 39767 consensus_queue.cc:695] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: daba39963d4346e1816089fd3a5442b5, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false W1031 23:03:53.615319 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 1 times. W1031 23:03:53.615708 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 1 times. I1031 23:03:53.616171 39777 consensus_queue.cc:695] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 0fc477287c154136abc1e2f3c82fb7f9, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:03:53.662752 10782 raft_consensus.cc:918] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 0f53a0d3ef7e44ebb0365c800752d5bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:03:53.697007 39737 consensus_queue.cc:695] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 044f1da9a27c46acb82b1386f829f4dc, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:03:53.697415 39737 consensus_queue.cc:695] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: daba39963d4346e1816089fd3a5442b5, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false W1031 23:03:54.115319 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 2 times. W1031 23:03:54.115813 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 2 times. W1031 23:03:54.534421 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 3 times. W1031 23:03:54.598920 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 3 times. W1031 23:03:55.034581 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 4 times. W1031 23:03:55.098960 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 4 times. W1031 23:03:55.534608 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 5 times. W1031 23:03:55.599126 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 5 times. W1031 23:03:56.034713 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 6 times. W1031 23:03:56.099160 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 6 times. W1031 23:03:56.534704 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 7 times. W1031 23:03:56.599225 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 7 times. I1031 23:03:56.728981 39746 raft_consensus.cc:411] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:03:56.729029 39746 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.620s I1031 23:03:56.729094 39746 raft_consensus.cc:435] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:03:56.729686 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:03:56.729717 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 W1031 23:03:56.729923 10718 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:03:56.729971 10719 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3eb30bd1bc34b95902fa61724239553: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:03:56.729995 10719 leader_election.cc:243] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:03:56.730281 39796 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.535s I1031 23:03:56.730304 39796 raft_consensus.cc:1908] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:03:56.794960 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:03:56.795014 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.371s I1031 23:03:56.795078 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:03:56.795598 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:03:56.795629 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:03:56.795820 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:03:56.795888 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:03:56.795936 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:03:56.796378 39799 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 3.349s I1031 23:03:56.796399 39799 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:03:57.034777 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 8 times. W1031 23:03:57.099297 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 8 times. W1031 23:03:57.534893 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 9 times. W1031 23:03:57.599354 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 9 times. W1031 23:03:58.034903 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 10 times. W1031 23:03:58.099488 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 10 times. W1031 23:03:58.535029 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 11 times. W1031 23:03:58.599509 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 11 times. W1031 23:03:59.035095 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 12 times. W1031 23:03:59.099568 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 12 times. W1031 23:03:59.535178 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 13 times. W1031 23:03:59.599617 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 13 times. W1031 23:04:00.035274 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 14 times. W1031 23:04:00.099686 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 14 times. I1031 23:04:00.200137 39746 raft_consensus.cc:411] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:00.200184 39746 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.087s I1031 23:04:00.200235 39746 raft_consensus.cc:435] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:04:00.201118 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:04:00.201150 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 W1031 23:04:00.201480 10718 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:00.201581 10719 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3eb30bd1bc34b95902fa61724239553: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:00.201608 10719 leader_election.cc:243] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:00.201884 39805 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.818s I1031 23:04:00.201918 39805 raft_consensus.cc:1908] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:00.535316 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 15 times. W1031 23:04:00.599848 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 15 times. W1031 23:04:01.035434 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 16 times. W1031 23:04:01.103385 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 16 times. W1031 23:04:01.535446 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 17 times. W1031 23:04:01.599958 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 17 times. W1031 23:04:02.035495 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 18 times. W1031 23:04:02.099973 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 18 times. I1031 23:04:02.236625 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:02.236677 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 3.919s I1031 23:04:02.236748 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:04:02.237401 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:04:02.237435 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:04:02.237697 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:02.237756 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:02.237782 10719 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:02.238093 40008 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 4.673s I1031 23:04:02.238114 40008 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:02.535588 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 19 times. W1031 23:04:02.600069 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 19 times. W1031 23:04:03.035730 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 20 times. W1031 23:04:03.100106 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 20 times. W1031 23:04:03.535722 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 21 times. W1031 23:04:03.600265 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 21 times. W1031 23:04:04.035776 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 22 times. W1031 23:04:04.100247 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 22 times. W1031 23:04:04.535836 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 23 times. W1031 23:04:04.600332 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 23 times. W1031 23:04:05.035967 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 24 times. I1031 23:04:05.045069 39746 raft_consensus.cc:411] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:05.045114 39746 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 4.411s I1031 23:04:05.045187 39746 raft_consensus.cc:435] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:04:05.045817 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:04:05.045853 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 W1031 23:04:05.046102 10718 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:05.046192 10719 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3eb30bd1bc34b95902fa61724239553: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:05.046216 10719 leader_election.cc:243] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:05.046479 40011 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.260s I1031 23:04:05.046502 40011 raft_consensus.cc:1908] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:05.100476 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 24 times. W1031 23:04:05.536001 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 25 times. W1031 23:04:05.600445 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 25 times. W1031 23:04:06.036021 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 26 times. W1031 23:04:06.100538 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 26 times. W1031 23:04:06.536134 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 27 times. W1031 23:04:06.600584 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 27 times. W1031 23:04:07.036154 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 28 times. W1031 23:04:07.100674 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 28 times. W1031 23:04:07.536186 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 29 times. W1031 23:04:07.600759 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 29 times. W1031 23:04:08.036330 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 30 times. W1031 23:04:08.100839 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 30 times. W1031 23:04:08.536520 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 31 times. W1031 23:04:08.600895 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 31 times. I1031 23:04:08.839357 39746 raft_consensus.cc:411] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:08.839421 39746 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 6.194s I1031 23:04:08.839505 39746 raft_consensus.cc:435] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:04:08.840272 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:04:08.840317 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 W1031 23:04:08.840518 10718 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:08.840641 10719 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3eb30bd1bc34b95902fa61724239553: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:08.840663 10719 leader_election.cc:243] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:08.840967 40019 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.999s I1031 23:04:08.841015 40019 raft_consensus.cc:1908] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:09.036470 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 32 times. W1031 23:04:09.100944 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 32 times. I1031 23:04:09.526410 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:09.526461 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 4.871s I1031 23:04:09.526520 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:04:09.527102 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:04:09.527142 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:04:09.527362 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:09.527416 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:09.527438 10719 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:09.527987 40020 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 5.770s I1031 23:04:09.528007 40020 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:09.536516 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 33 times. W1031 23:04:09.601023 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 33 times. W1031 23:04:10.036630 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 34 times. W1031 23:04:10.101125 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 34 times. W1031 23:04:10.536582 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 35 times. W1031 23:04:10.601157 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 35 times. W1031 23:04:11.036775 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 36 times. W1031 23:04:11.101235 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 36 times. W1031 23:04:11.536840 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 37 times. W1031 23:04:11.601395 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 37 times. W1031 23:04:12.036926 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 38 times. W1031 23:04:12.101408 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 38 times. W1031 23:04:12.536989 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 39 times. W1031 23:04:12.601490 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 39 times. W1031 23:04:13.037089 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 40 times. W1031 23:04:13.101524 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 40 times. I1031 23:04:13.372252 39746 raft_consensus.cc:411] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:13.372303 39746 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 15.256s I1031 23:04:13.372357 39746 raft_consensus.cc:435] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:04:13.372962 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:04:13.373005 39746 leader_election.cc:216] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 W1031 23:04:13.373327 10718 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:13.373431 10719 leader_election.cc:278] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer d3eb30bd1bc34b95902fa61724239553: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:13.373455 10719 leader_election.cc:243] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:13.373780 40050 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 12.866s I1031 23:04:13.373800 40050 raft_consensus.cc:1908] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:13.536978 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 41 times. W1031 23:04:13.601626 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 41 times. W1031 23:04:14.037139 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 42 times. W1031 23:04:14.101733 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 42 times. W1031 23:04:14.537221 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 43 times. W1031 23:04:14.601874 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 43 times. W1031 23:04:15.037297 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 44 times. W1031 23:04:15.101827 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 44 times. W1031 23:04:15.537286 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 45 times. W1031 23:04:15.601927 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 45 times. W1031 23:04:16.037384 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 46 times. W1031 23:04:16.101979 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 46 times. W1031 23:04:16.537470 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 47 times. W1031 23:04:16.602104 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 47 times. I1031 23:04:17.001978 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:17.002024 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 4.134s I1031 23:04:17.002084 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:04:17.002735 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:04:17.002771 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:04:17.002948 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:17.003013 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:17.003029 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:17.003412 40058 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 10.758s I1031 23:04:17.003434 40058 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:17.037586 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 48 times. W1031 23:04:17.102193 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 48 times. I1031 23:04:17.288919 10776 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 10.197s I1031 23:04:17.288964 10776 raft_consensus.cc:1747] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d3eb30bd1bc34b95902fa61724239553 in term 0. I1031 23:04:17.290776 10776 raft_consensus.cc:2206] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:04:17.290815 10776 raft_consensus.cc:2162] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 9.710s I1031 23:04:17.291864 10776 raft_consensus.cc:1747] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d3eb30bd1bc34b95902fa61724239553 in term 1. W1031 23:04:17.537662 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 49 times. W1031 23:04:17.602241 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 49 times. I1031 23:04:17.794100 10774 raft_consensus.cc:918] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer d3eb30bd1bc34b95902fa61724239553: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) W1031 23:04:18.037680 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 50 times. W1031 23:04:18.102329 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 50 times. W1031 23:04:18.537753 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 51 times. W1031 23:04:18.602383 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 51 times. W1031 23:04:19.037824 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 52 times. W1031 23:04:19.102542 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 52 times. W1031 23:04:19.537889 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 53 times. W1031 23:04:19.602557 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 53 times. W1031 23:04:20.037886 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 54 times. W1031 23:04:20.102622 10720 consensus_peers.cc:357] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 044f1da9a27c46acb82b1386f829f4dc (c3-kudu-tst-st03.bj:18700): Couldn't send request to peer 044f1da9a27c46acb82b1386f829f4dc for tablet feeb15ca2b1f490296ccd8a8edacf49f. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 54 times. W1031 23:04:20.538106 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 55 times. I1031 23:04:20.603268 39767 consensus_queue.cc:695] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 044f1da9a27c46acb82b1386f829f4dc, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false W1031 23:04:21.038125 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 56 times. W1031 23:04:21.538199 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 57 times. W1031 23:04:22.038242 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 58 times. I1031 23:04:22.477262 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.477314 10764 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.477327 10767 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.477394 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.477849 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.477864 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.477869 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.478091 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.478109 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.478328 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.478402 10764 tablet_service.cc:650] Processing CreateTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171024 <= VALUES < 20171025 I1031 23:04:22.478471 10757 tablet_service.cc:650] Processing CreateTablet for tablet 117cd74bcdab4942901acb0005cdd702 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171026 <= VALUES < 20171027 I1031 23:04:22.478480 10767 tablet_service.cc:650] Processing CreateTablet for tablet 802f72af09384e95acd389f46a1d9c84 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171027 <= VALUES < 20171028 I1031 23:04:22.478590 10754 tablet_service.cc:650] Processing CreateTablet for tablet 9710e0187e434acfa54ca8f8504434ed (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 2, RANGE (date) PARTITION 20171030 <= VALUES < 20171031 I1031 23:04:22.478590 10763 tablet_service.cc:650] Processing CreateTablet for tablet 46fc46e028434bc69f467922a9a59101 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171030 <= VALUES < 20171031 I1031 23:04:22.478636 10769 tablet_service.cc:650] Processing CreateTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171023 <= VALUES < 20171024 I1031 23:04:22.478766 10759 tablet_service.cc:650] Processing CreateTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171025 <= VALUES < 20171026 I1031 23:04:22.478768 10762 tablet_service.cc:650] Processing CreateTablet for tablet 582c1281bf5147b2bd40c9a950a1152f (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171027 <= VALUES < 20171028 I1031 23:04:22.478816 10770 tablet_service.cc:650] Processing CreateTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171029 <= VALUES < 20171030 I1031 23:04:22.496003 10756 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.500430 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.511821 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.516897 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.519281 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.520567 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.524905 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.532268 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.535634 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.538003 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:22.538302 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 59 times. I1031 23:04:22.542871 10756 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.561713 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.567147 10756 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.567176 10758 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.577854 10758 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.582702 10758 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.582995 10770 ts_tablet_manager.cc:862] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.600294 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.602711 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.608417 10759 ts_tablet_manager.cc:862] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.608664 10759 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.608676 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.634307 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.634570 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.643776 10754 ts_tablet_manager.cc:862] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.647166 10754 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.659833 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.659844 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.675906 10769 ts_tablet_manager.cc:862] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.681313 10769 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.690119 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.692498 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.700873 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.701087 10763 ts_tablet_manager.cc:862] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.708931 10763 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.726243 10762 ts_tablet_manager.cc:862] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.751425 10767 ts_tablet_manager.cc:862] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.775483 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.776751 10757 ts_tablet_manager.cc:862] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.783121 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.800245 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.801750 10764 ts_tablet_manager.cc:862] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:22.806450 10764 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.814853 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.844007 10764 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.849715 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.849954 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.861328 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:22.871737 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:23.038404 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 60 times. I1031 23:04:23.040905 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.055428 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.066984 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.076866 10764 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.107985 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.113699 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.113715 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.128357 10757 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.129655 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.131952 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:23.538453 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 61 times. I1031 23:04:23.571605 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.593034 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.599439 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.610819 10757 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.635274 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.636618 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.637902 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.645202 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.680560 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:23.682945 10757 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:24.038489 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 62 times. W1031 23:04:24.538601 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 63 times. I1031 23:04:24.629199 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.629199 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.659452 10764 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.671859 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.683622 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.695050 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.708221 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.717597 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.722966 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:24.730315 10764 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:25.038725 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 64 times. W1031 23:04:25.538725 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 65 times. W1031 23:04:26.038807 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 66 times. W1031 23:04:26.538884 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 67 times. I1031 23:04:26.708401 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.721897 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.754855 10764 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.755132 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.757556 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.757581 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.763008 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.793792 10757 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.804172 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:26.807538 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:27.038925 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 68 times. W1031 23:04:27.539088 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 69 times. W1031 23:04:28.039214 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 70 times. W1031 23:04:28.539166 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 71 times. W1031 23:04:29.039345 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 72 times. W1031 23:04:29.539363 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 73 times. I1031 23:04:29.780344 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:29.780409 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 6.173s I1031 23:04:29.780486 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:04:29.781242 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:04:29.781286 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:04:29.781517 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:29.781647 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:29.781704 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:29.782068 40090 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 15.874s I1031 23:04:29.782091 40090 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:30.039361 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 74 times. W1031 23:04:30.539429 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 75 times. I1031 23:04:30.830740 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.849309 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.858687 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.872097 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.874482 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.886735 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.896533 10757 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.908963 10757 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.910276 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:30.937783 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:31.039484 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 76 times. W1031 23:04:31.539582 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 77 times. W1031 23:04:32.039641 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 78 times. W1031 23:04:32.539644 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 79 times. W1031 23:04:33.039758 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 80 times. W1031 23:04:33.539829 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 81 times. W1031 23:04:34.039924 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 82 times. W1031 23:04:34.540071 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 83 times. W1031 23:04:35.040168 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 84 times. W1031 23:04:35.540138 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 85 times. W1031 23:04:36.040194 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 86 times. W1031 23:04:36.540334 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 87 times. W1031 23:04:37.040288 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 88 times. W1031 23:04:37.540387 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 89 times. W1031 23:04:38.040415 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 90 times. W1031 23:04:38.540453 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 91 times. W1031 23:04:39.040735 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 92 times. I1031 23:04:39.060506 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.070008 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.078435 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.083845 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.100306 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.103667 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.127699 10757 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.138739 10757 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.147110 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:39.149487 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:39.540591 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 93 times. W1031 23:04:40.040676 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 94 times. W1031 23:04:40.540846 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 95 times. W1031 23:04:41.040894 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 96 times. W1031 23:04:41.540917 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 97 times. W1031 23:04:42.041013 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 98 times. W1031 23:04:42.541105 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 99 times. W1031 23:04:43.041074 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 100 times. W1031 23:04:43.541190 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 101 times. W1031 23:04:44.041254 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 102 times. W1031 23:04:44.541373 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 103 times. W1031 23:04:45.041417 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 104 times. W1031 23:04:45.541510 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 105 times. W1031 23:04:46.041548 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 106 times. W1031 23:04:46.541589 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 107 times. W1031 23:04:47.041677 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 108 times. I1031 23:04:47.187480 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:47.187525 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 10.899s I1031 23:04:47.187582 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:04:47.188225 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:04:47.188262 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:04:47.188468 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:47.188505 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:47.188529 10719 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:47.188819 40153 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 3.533s I1031 23:04:47.188863 40153 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:04:47.541798 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 109 times. W1031 23:04:48.041862 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 110 times. W1031 23:04:48.541829 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 111 times. W1031 23:04:49.041940 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 112 times. W1031 23:04:49.542102 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 113 times. W1031 23:04:50.042153 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 114 times. W1031 23:04:50.542155 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 115 times. W1031 23:04:51.042292 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 116 times. W1031 23:04:51.542305 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 117 times. W1031 23:04:52.042438 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 118 times. I1031 23:04:52.501969 10757 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.502496 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.502514 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.502553 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.502562 10764 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.503113 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.503113 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.503137 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.503288 10767 tablet_service.cc:650] Processing CreateTablet for tablet acb401e40e7049fbbfdbeb40474353d9 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171026 <= VALUES < 20171027 I1031 23:04:52.503289 10757 tablet_service.cc:650] Processing CreateTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171024 <= VALUES < 20171025 I1031 23:04:52.503289 10764 tablet_service.cc:650] Processing CreateTablet for tablet 721e8a57234447eabda00f5df65bd574 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 1, RANGE (date) PARTITION 20171027 <= VALUES < 20171028 I1031 23:04:52.503486 10769 tablet_service.cc:650] Processing CreateTablet for tablet de37a68992504da8a4640f8ab4653544 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171025 <= VALUES < 20171026 I1031 23:04:52.503526 10762 tablet_service.cc:650] Processing CreateTablet for tablet 8356b43879104775b9a8c5a9e546e85b (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171023 <= VALUES < 20171024 I1031 23:04:52.503526 10763 tablet_service.cc:650] Processing CreateTablet for tablet 32984b820db347e1b2d4d0786cf221f2 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171028 <= VALUES < 20171029 I1031 23:04:52.503532 10770 tablet_service.cc:650] Processing CreateTablet for tablet 0def0264c0db4c438e16c22b23a0b43a (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171027 <= VALUES < 20171028 I1031 23:04:52.503578 10754 tablet_service.cc:650] Processing CreateTablet for tablet 1cebccaea10d49478c004cca39676556 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 4, RANGE (date) PARTITION 20171029 <= VALUES < 20171030 I1031 23:04:52.520624 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.524062 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.529583 10759 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.530865 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.540279 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.541612 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:52.542562 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 119 times. I1031 23:04:52.548449 10759 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.553871 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.558233 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.575002 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.584769 10759 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.602110 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.602113 10758 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.603312 10763 ts_tablet_manager.cc:862] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:52.606686 10763 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.613062 10763 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.628661 10762 ts_tablet_manager.cc:862] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:52.628937 10762 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.653673 10769 ts_tablet_manager.cc:862] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:52.656083 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.660487 10769 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.663241 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.678952 10764 ts_tablet_manager.cc:862] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:52.683337 10764 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.683353 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.700606 10769 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.704125 10754 ts_tablet_manager.cc:862] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:52.711511 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.712852 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.729398 10757 ts_tablet_manager.cc:862] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:52.754570 10767 ts_tablet_manager.cc:862] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:52.769057 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:04:52.769096 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 5.255s I1031 23:04:52.769158 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:04:52.769721 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:04:52.769752 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:04:52.769946 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:04:52.770032 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:04:52.770090 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:04:52.770426 40179 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 6.426s I1031 23:04:52.770444 40179 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:04:52.779811 10770 ts_tablet_manager.cc:862] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:04:52.802667 10770 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.829036 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.839679 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.843042 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.852819 10770 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.856170 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.862501 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:52.879346 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:53.042461 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 120 times. I1031 23:04:53.089269 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.092686 10770 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.111438 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.118079 10770 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.137583 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.139943 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.150269 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.169644 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:53.542543 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 121 times. I1031 23:04:53.609027 10770 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.640894 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.647303 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.659687 10770 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.661006 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.663348 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.687217 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:53.693709 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:54.042704 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 122 times. W1031 23:04:54.542737 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 123 times. I1031 23:04:54.634805 10770 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:54.674967 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:54.695997 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:54.702461 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:54.722966 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:54.728382 10770 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:54.729718 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:54.749686 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:55.042811 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 124 times. I1031 23:04:55.469619 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:55.473002 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:55.475312 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:55.512116 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:55.523089 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:55.527797 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:55.542860 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 125 times. I1031 23:04:55.545289 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:55.549638 10770 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:55.556984 10770 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:55.582789 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:56.042920 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 126 times. W1031 23:04:56.543007 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 127 times. I1031 23:04:56.711944 10770 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:56.747778 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:56.751157 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:56.753819 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:56.784749 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:56.786049 10770 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:56.803417 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:04:56.834487 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:04:57.043066 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 128 times. W1031 23:04:57.543200 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 129 times. W1031 23:04:58.043210 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 130 times. W1031 23:04:58.543375 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 131 times. W1031 23:04:59.043385 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 132 times. W1031 23:04:59.543414 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 133 times. W1031 23:05:00.043484 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 134 times. W1031 23:05:00.543512 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 135 times. I1031 23:05:00.823220 10770 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:00.857717 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:00.881616 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:00.886018 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:00.898301 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:00.911676 10770 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:00.925875 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:00.925917 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 17.608s I1031 23:05:00.925982 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:00.926609 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:00.926652 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:00.926916 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:00.926980 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:00.927031 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:00.927382 40194 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 15.348s I1031 23:05:00.927402 40194 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:00.938199 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:00.949350 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:01.043589 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 136 times. W1031 23:05:01.543681 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 137 times. W1031 23:05:02.043728 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 138 times. W1031 23:05:02.543961 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 139 times. W1031 23:05:03.043908 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 140 times. W1031 23:05:03.543951 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 141 times. W1031 23:05:04.044044 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 142 times. W1031 23:05:04.544093 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 143 times. W1031 23:05:05.044158 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 144 times. W1031 23:05:05.544212 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 145 times. W1031 23:05:06.044275 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 146 times. W1031 23:05:06.544472 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 147 times. W1031 23:05:07.044437 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 148 times. W1031 23:05:07.544569 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 149 times. W1031 23:05:08.044559 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 150 times. W1031 23:05:08.544668 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 151 times. I1031 23:05:09.043673 10770 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:09.044703 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 152 times. I1031 23:05:09.051110 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:09.091692 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:09.097550 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:09.102917 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:09.132405 10770 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:09.157359 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:09.161805 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:09.544767 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 153 times. W1031 23:05:10.044805 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 154 times. W1031 23:05:10.544924 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 155 times. W1031 23:05:11.044980 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 156 times. W1031 23:05:11.545094 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 157 times. W1031 23:05:12.045135 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 158 times. W1031 23:05:12.545213 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 159 times. W1031 23:05:13.045259 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 160 times. W1031 23:05:13.545447 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 161 times. W1031 23:05:14.045359 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 162 times. W1031 23:05:14.545459 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 163 times. W1031 23:05:15.045511 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 164 times. W1031 23:05:15.545608 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 165 times. W1031 23:05:16.045655 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 166 times. W1031 23:05:16.545697 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 167 times. W1031 23:05:17.045732 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 168 times. W1031 23:05:17.545778 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 169 times. W1031 23:05:18.045923 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 170 times. I1031 23:05:18.097091 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:18.097146 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 4.301s I1031 23:05:18.097204 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:18.097795 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:18.097823 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:18.098045 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:18.098117 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:18.098150 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:18.098493 40447 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 11.330s I1031 23:05:18.098518 40447 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:18.546073 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 171 times. W1031 23:05:19.046082 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 172 times. W1031 23:05:19.546236 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 173 times. W1031 23:05:20.046180 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 174 times. W1031 23:05:20.546239 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 175 times. W1031 23:05:21.046314 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 176 times. W1031 23:05:21.546388 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 177 times. W1031 23:05:22.046448 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 178 times. I1031 23:05:22.526214 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.526367 10770 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.526407 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.526948 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.526953 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.526986 10757 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.527034 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.527051 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.527302 10770 tablet_service.cc:650] Processing CreateTablet for tablet 47eeb8448c454968a45bf1a984427b25 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171029 <= VALUES < 20171030 I1031 23:05:22.527310 10754 tablet_service.cc:650] Processing CreateTablet for tablet f2af422919fd4560bd6441509a68d64b (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171028 <= VALUES < 20171029 I1031 23:05:22.527504 10757 tablet_service.cc:650] Processing CreateTablet for tablet 29fdcf1d33f94660b0c77445ec7e559b (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171023 <= VALUES < 20171024 I1031 23:05:22.527534 10767 tablet_service.cc:650] Processing CreateTablet for tablet eeb3a3cb417b466abffd16749d165a61 (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 3, RANGE (date) PARTITION 20171028 <= VALUES < 20171029 I1031 23:05:22.544839 10769 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:22.546525 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 179 times. I1031 23:05:22.547382 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.552212 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.565085 10769 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.569473 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.575803 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.582145 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.583423 10769 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.587132 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.606091 10767 ts_tablet_manager.cc:862] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:05:22.608451 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.622890 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.627449 10767 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.629772 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.630904 10754 ts_tablet_manager.cc:862] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:05:22.633270 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.639027 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.656057 10757 ts_tablet_manager.cc:862] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:05:22.658381 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.680138 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.681351 10770 ts_tablet_manager.cc:862] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:05:22.688678 10770 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.706364 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.706373 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.725252 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.727718 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.733616 10757 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.733640 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.835528 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.842516 10770 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.842545 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.873705 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.883134 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.885495 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.897852 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:22.897874 10770 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:23.046556 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 180 times. I1031 23:05:23.126863 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.130262 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.139827 10770 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.160689 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.174007 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.176321 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.184677 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.195071 10770 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:23.546701 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 181 times. I1031 23:05:23.643368 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.654264 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.689599 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.691965 10770 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.696696 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.712100 10770 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.718626 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:23.742213 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:24.046807 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 182 times. W1031 23:05:24.546859 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 183 times. I1031 23:05:24.668614 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:24.686048 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:24.746903 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:24.748241 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:24.748293 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:24.752720 10757 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:24.758092 10757 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:24.779470 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:25.046833 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 184 times. I1031 23:05:25.450023 10757 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:25.455395 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:25.491783 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:25.508617 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:25.508623 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:25.543300 10757 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:25.546919 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 185 times. I1031 23:05:25.560608 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:25.569967 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:26.047000 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 186 times. W1031 23:05:26.547075 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 187 times. I1031 23:05:26.735667 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:26.737046 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:26.809104 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:26.810459 10757 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:26.820845 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:26.832283 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:26.835682 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:26.847167 10757 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:27.047109 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 188 times. W1031 23:05:27.547163 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 189 times. W1031 23:05:28.047246 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 190 times. I1031 23:05:28.247858 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.270355 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.283990 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.286342 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.300766 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.304083 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.317452 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.327837 10757 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.355024 10757 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:28.383015 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:28.547297 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 191 times. W1031 23:05:29.047323 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 192 times. W1031 23:05:29.547415 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 193 times. W1031 23:05:30.047565 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 194 times. W1031 23:05:30.547569 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 195 times. I1031 23:05:30.832862 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:30.839375 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:30.923409 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:30.933960 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:30.939344 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:30.956255 10757 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:30.959621 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:30.962965 10757 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:31.036417 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:31.036476 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 14.451s I1031 23:05:31.036558 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:31.037279 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:31.037322 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:31.037515 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:31.037564 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:31.037593 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:31.037852 40482 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 5.340s I1031 23:05:31.037899 40482 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:31.047680 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 196 times. W1031 23:05:31.547781 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 197 times. W1031 23:05:32.047848 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 198 times. W1031 23:05:32.547941 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 199 times. W1031 23:05:33.048099 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 200 times. W1031 23:05:33.548007 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 201 times. I1031 23:05:33.890697 39743 ts_tablet_manager.cc:755] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.223s user 0.000s sys 0.001s W1031 23:05:33.890741 39743 ts_tablet_manager.cc:785] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101222ms W1031 23:05:33.890863 39743 ts_tablet_manager.cc:787] T c7eb286102a648969b5cf2a93710a72c P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451106 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451118 (+ 12us) service_pool.cc:202] Handling call 1031 23:03:52.451270 (+ 152us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451273 (+ 3us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.664491 (+213218us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.665988 (+ 1497us) inbound_call.cc:130] Queueing success response 1031 23:03:52.666037 (+ 49us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.668063 (+ 2026us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.668081 (+ 18us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.668348 (+ 267us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.668352 (+ 4us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.668353 (+ 1us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.668353 (+ 0us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":111360,"mutex_wait_us":101221446,"raft.queue_time_us":317,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":4,"tablet-open.queue_time_us":705,"thread_start_us":1340,"threads_started":3} I1031 23:05:33.890913 39743 ts_tablet_manager.cc:723] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.891420 39743 tablet_bootstrap.cc:382] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.891582 39743 tablet_bootstrap.cc:541] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.891728 39743 tablet_bootstrap.cc:484] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.892092 39728 ts_tablet_manager.cc:755] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.301s user 0.001s sys 0.000s W1031 23:05:33.892119 39728 ts_tablet_manager.cc:785] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101301ms W1031 23:05:33.892226 39728 ts_tablet_manager.cc:787] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451292 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451306 (+ 14us) service_pool.cc:202] Handling call 1031 23:03:52.451379 (+ 73us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451383 (+ 4us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.588575 (+137192us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.589446 (+ 871us) inbound_call.cc:130] Queueing success response 1031 23:03:52.589551 (+ 105us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.591021 (+ 1470us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.591039 (+ 18us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.591306 (+ 267us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.591311 (+ 5us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.591312 (+ 1us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.591312 (+ 0us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":111445,"mutex_wait_us":101299868,"raft.queue_time_us":336,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":3,"tablet-open.queue_time_us":347,"thread_start_us":919,"threads_started":3} I1031 23:05:33.892266 39728 ts_tablet_manager.cc:723] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.892668 39728 tablet_bootstrap.cc:382] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.892761 39743 tablet_bootstrap.cc:382] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.892791 39743 ts_tablet_manager.cc:728] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s I1031 23:05:33.892802 39728 tablet_bootstrap.cc:541] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.892920 39728 tablet_bootstrap.cc:484] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.893381 39748 ts_tablet_manager.cc:755] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.201s user 0.001s sys 0.000s W1031 23:05:33.893400 39748 ts_tablet_manager.cc:785] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101200ms W1031 23:05:33.893499 39748 ts_tablet_manager.cc:787] T 6ca74226a6af40e2a78046fc3591c038 P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451337 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451353 (+ 16us) service_pool.cc:202] Handling call 1031 23:03:52.451429 (+ 76us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451432 (+ 3us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.689799 (+238367us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.690769 (+ 970us) inbound_call.cc:130] Queueing success response 1031 23:03:52.690809 (+ 40us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.692804 (+ 1995us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.692826 (+ 22us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.693170 (+ 344us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.693178 (+ 8us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.693179 (+ 1us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.693179 (+ 0us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":111392,"mutex_wait_us":101199411,"raft.queue_time_us":138,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"tablet-open.queue_time_us":268,"tcmalloc_contention_cycles":1664,"thread_start_us":787,"threads_started":3} I1031 23:05:33.893541 39748 ts_tablet_manager.cc:723] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.893905 39748 tablet_bootstrap.cc:382] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.894039 39748 tablet_bootstrap.cc:541] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.894158 39748 tablet_bootstrap.cc:484] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.894165 39743 raft_consensus.cc:288] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } I1031 23:05:33.894189 39743 raft_consensus.cc:319] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.894207 39743 raft_consensus.cc:548] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.894229 39743 consensus_queue.cc:176] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.894314 39763 ts_tablet_manager.cc:755] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.118s user 0.001s sys 0.000s W1031 23:05:33.894332 39763 ts_tablet_manager.cc:785] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101117ms W1031 23:05:33.894428 39763 ts_tablet_manager.cc:787] T feeb15ca2b1f490296ccd8a8edacf49f P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451277 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451368 (+ 91us) service_pool.cc:202] Handling call 1031 23:03:52.451421 (+ 53us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451424 (+ 3us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.774217 (+322793us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.774995 (+ 778us) inbound_call.cc:130] Queueing success response 1031 23:03:52.775054 (+ 59us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.776690 (+ 1636us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.776710 (+ 20us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.777000 (+ 290us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.777004 (+ 4us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.777005 (+ 1us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.777005 (+ 0us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":111331,"mutex_wait_us":101116868,"raft.queue_time_us":123,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"tablet-open.queue_time_us":307,"thread_start_us":509,"threads_started":3} I1031 23:05:33.894464 39763 ts_tablet_manager.cc:723] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.894799 39763 tablet_bootstrap.cc:382] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.894923 39763 tablet_bootstrap.cc:541] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.895020 39763 tablet_bootstrap.cc:484] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.895475 39728 tablet_bootstrap.cc:382] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.895476 39738 ts_tablet_manager.cc:755] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.253s user 0.000s sys 0.001s I1031 23:05:33.895505 39728 ts_tablet_manager.cc:728] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s W1031 23:05:33.895515 39738 ts_tablet_manager.cc:785] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101252ms I1031 23:05:33.895519 39768 ts_tablet_manager.cc:755] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.093s user 0.000s sys 0.000s W1031 23:05:33.895551 39768 ts_tablet_manager.cc:785] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101092ms I1031 23:05:33.895546 39733 ts_tablet_manager.cc:755] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.262s user 0.000s sys 0.002s W1031 23:05:33.895587 39733 ts_tablet_manager.cc:785] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101262ms I1031 23:05:33.895673 39743 ts_tablet_manager.cc:755] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s W1031 23:05:33.895699 39768 ts_tablet_manager.cc:787] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451093 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451108 (+ 15us) service_pool.cc:202] Handling call 1031 23:03:52.451250 (+ 142us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451265 (+ 15us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.799546 (+348281us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.800533 (+ 987us) inbound_call.cc:130] Queueing success response 1031 23:03:52.800588 (+ 55us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.802702 (+ 2114us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.802720 (+ 18us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.802976 (+ 256us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.802980 (+ 4us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.802981 (+ 1us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.802981 (+ 0us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":111538,"mutex_wait_us":101092120,"raft.queue_time_us":120,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":3,"tablet-open.queue_time_us":256,"tcmalloc_contention_cycles":2048,"thread_start_us":439,"threads_started":3} I1031 23:05:33.895717 39758 ts_tablet_manager.cc:755] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.152s user 0.000s sys 0.001s W1031 23:05:33.895741 39758 ts_tablet_manager.cc:785] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101152ms I1031 23:05:33.895746 39753 ts_tablet_manager.cc:755] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 101.178s user 0.001s sys 0.000s W1031 23:05:33.895737 39738 ts_tablet_manager.cc:787] T fcde65c4e4cf4df29b9ef9884ce292b2 P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451438 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451617 (+ 179us) service_pool.cc:202] Handling call 1031 23:03:52.451682 (+ 65us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451685 (+ 3us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.639198 (+187513us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.640456 (+ 1258us) inbound_call.cc:130] Queueing success response 1031 23:03:52.640526 (+ 70us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.642764 (+ 2238us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.642783 (+ 19us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.643105 (+ 322us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.643109 (+ 4us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.643109 (+ 0us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.643110 (+ 1us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":110698,"mutex_wait_us":101251345,"raft.queue_time_us":338,"raft.run_cpu_time_us":14,"raft.run_wall_time_us":14,"tablet-open.queue_time_us":521,"thread_start_us":1186,"threads_started":3} W1031 23:05:33.895763 39753 ts_tablet_manager.cc:785] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 101177ms W1031 23:05:33.895854 39733 ts_tablet_manager.cc:787] T 269d7d8c1d21484ab7f554a011190a84 P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451356 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451371 (+ 15us) service_pool.cc:202] Handling call 1031 23:03:52.451454 (+ 83us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451458 (+ 4us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.613908 (+162450us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.614866 (+ 958us) inbound_call.cc:130] Queueing success response 1031 23:03:52.614905 (+ 39us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.633469 (+ 18564us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.633497 (+ 28us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.633987 (+ 490us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.633994 (+ 7us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.633995 (+ 1us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.633996 (+ 1us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":110716,"mutex_wait_us":101260329,"raft.queue_time_us":462,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"tablet-open.queue_time_us":393,"tcmalloc_contention_cycles":89728,"thread_start_us":1323,"threads_started":3} W1031 23:05:33.895887 39758 ts_tablet_manager.cc:787] T 318722d72b774164aba6657098b58c90 P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451169 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451180 (+ 11us) service_pool.cc:202] Handling call 1031 23:03:52.451288 (+ 108us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451291 (+ 3us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.740471 (+289180us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.741391 (+ 920us) inbound_call.cc:130] Queueing success response 1031 23:03:52.741431 (+ 40us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.743382 (+ 1951us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.743399 (+ 17us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.743724 (+ 325us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.743729 (+ 5us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.743730 (+ 1us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.743730 (+ 0us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":111001,"mutex_wait_us":101151511,"raft.queue_time_us":107,"raft.run_cpu_time_us":3,"raft.run_wall_time_us":3,"tablet-open.queue_time_us":222,"thread_start_us":420,"threads_started":3} W1031 23:05:33.895895 39753 ts_tablet_manager.cc:787] T f05a7aa04e784d02a30ce666139a7263 P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:03:52.451242 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:03:52.451252 (+ 10us) service_pool.cc:202] Handling call 1031 23:03:52.451318 (+ 66us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:03:52.451320 (+ 2us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:03:52.715153 (+263833us) tablet_metadata.cc:461] Metadata flushed 1031 23:03:52.716194 (+ 1041us) inbound_call.cc:130] Queueing success response 1031 23:03:52.716257 (+ 63us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:03:52.718085 (+ 1828us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:03:52.718106 (+ 21us) tablet_peer.cc:161] Creating consensus instance 1031 23:03:52.718433 (+ 327us) tablet_peer.cc:183] Starting instrumentation 1031 23:03:52.718439 (+ 6us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:03:52.718439 (+ 0us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:03:52.718440 (+ 1us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":111185,"mutex_wait_us":101176368,"raft.queue_time_us":114,"raft.run_cpu_time_us":5,"raft.run_wall_time_us":5,"tablet-open.queue_time_us":229,"thread_start_us":908,"threads_started":3} I1031 23:05:33.895889 39748 tablet_bootstrap.cc:382] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.895920 39748 ts_tablet_manager.cc:728] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s I1031 23:05:33.895934 39743 ts_tablet_manager.cc:723] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.895943 39758 ts_tablet_manager.cc:723] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.895967 39733 ts_tablet_manager.cc:723] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.895987 39738 ts_tablet_manager.cc:723] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.896021 39768 ts_tablet_manager.cc:723] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.896041 39753 ts_tablet_manager.cc:723] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.896178 39743 tablet_bootstrap.cc:382] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.896193 39728 raft_consensus.cc:288] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.896214 39728 raft_consensus.cc:319] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.896224 39728 raft_consensus.cc:548] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.896246 39728 consensus_queue.cc:176] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.896286 39743 tablet_bootstrap.cc:541] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.896383 39743 tablet_bootstrap.cc:484] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.896456 39768 tablet_bootstrap.cc:382] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.896476 39733 tablet_bootstrap.cc:382] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.896518 39753 tablet_bootstrap.cc:382] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.896530 39758 tablet_bootstrap.cc:382] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.896535 39738 tablet_bootstrap.cc:382] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.896567 39768 tablet_bootstrap.cc:541] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.896610 39733 tablet_bootstrap.cc:541] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.896667 39768 tablet_bootstrap.cc:484] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.896673 39738 tablet_bootstrap.cc:541] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.896786 39733 tablet_bootstrap.cc:484] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.896836 39738 tablet_bootstrap.cc:484] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.899183 39763 tablet_bootstrap.cc:382] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.899227 39763 ts_tablet_manager.cc:728] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s I1031 23:05:33.899325 39753 tablet_bootstrap.cc:541] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.899410 39728 ts_tablet_manager.cc:755] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s I1031 23:05:33.899428 39753 tablet_bootstrap.cc:484] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.899440 39748 raft_consensus.cc:288] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } I1031 23:05:33.899468 39748 raft_consensus.cc:319] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.899477 39758 tablet_bootstrap.cc:541] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.899482 39748 raft_consensus.cc:548] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.899508 39748 consensus_queue.cc:176] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.899600 39758 tablet_bootstrap.cc:484] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.899767 39748 ts_tablet_manager.cc:755] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s I1031 23:05:33.899869 39728 ts_tablet_manager.cc:723] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.899878 39748 ts_tablet_manager.cc:723] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.900231 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.900235 39748 tablet_bootstrap.cc:382] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.900239 39728 tablet_bootstrap.cc:382] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.900264 10757 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 5110f0095bd94f67abf0aad906332efe I1031 23:05:33.900279 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(5110f0095bd94f67abf0aad906332efe) I1031 23:05:33.900296 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(5110f0095bd94f67abf0aad906332efe) I1031 23:05:33.900306 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(5110f0095bd94f67abf0aad906332efe) I1031 23:05:33.900318 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(5110f0095bd94f67abf0aad906332efe) I1031 23:05:33.900346 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(5110f0095bd94f67abf0aad906332efe) I1031 23:05:33.900359 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(5110f0095bd94f67abf0aad906332efe) I1031 23:05:33.900378 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(5110f0095bd94f67abf0aad906332efe) I1031 23:05:33.900394 10757 raft_consensus.cc:1569] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.900405 39728 tablet_bootstrap.cc:541] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.900413 10757 raft_consensus.cc:1585] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.900409 39748 tablet_bootstrap.cc:541] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.900545 39728 tablet_bootstrap.cc:484] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.900598 39748 tablet_bootstrap.cc:484] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.901968 39768 tablet_bootstrap.cc:382] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.902009 39768 ts_tablet_manager.cc:728] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s I1031 23:05:33.902014 39743 tablet_bootstrap.cc:382] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.902050 39743 ts_tablet_manager.cc:728] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s I1031 23:05:33.902117 39763 raft_consensus.cc:288] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } I1031 23:05:33.902151 39763 raft_consensus.cc:319] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.902173 39763 raft_consensus.cc:548] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.902201 39763 consensus_queue.cc:176] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.902786 39738 tablet_bootstrap.cc:382] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.902823 39738 ts_tablet_manager.cc:728] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s I1031 23:05:33.902882 39768 raft_consensus.cc:288] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } I1031 23:05:33.902912 39768 raft_consensus.cc:319] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.902920 39743 raft_consensus.cc:288] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.902930 39768 raft_consensus.cc:548] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.902932 39763 ts_tablet_manager.cc:755] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s I1031 23:05:33.902941 39743 raft_consensus.cc:319] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.902951 39733 tablet_bootstrap.cc:382] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.902952 39768 consensus_queue.cc:176] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.902953 39743 raft_consensus.cc:548] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.902987 39733 ts_tablet_manager.cc:728] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.001s I1031 23:05:33.903015 39743 consensus_queue.cc:176] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.903949 39763 ts_tablet_manager.cc:723] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.904289 39763 tablet_bootstrap.cc:382] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.904424 39763 tablet_bootstrap.cc:541] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.904547 39763 tablet_bootstrap.cc:484] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.905318 39768 ts_tablet_manager.cc:755] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s I1031 23:05:33.905360 39768 ts_tablet_manager.cc:723] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.905467 39753 tablet_bootstrap.cc:382] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.905498 39753 ts_tablet_manager.cc:728] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s I1031 23:05:33.905519 39758 tablet_bootstrap.cc:382] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.905536 39743 ts_tablet_manager.cc:755] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s I1031 23:05:33.905549 39758 ts_tablet_manager.cc:728] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.000s I1031 23:05:33.905570 39743 ts_tablet_manager.cc:723] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.905604 39738 raft_consensus.cc:288] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.905634 39738 raft_consensus.cc:319] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.905642 39768 tablet_bootstrap.cc:382] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.905637 10757 ts_tablet_manager.cc:1020] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.905658 39738 raft_consensus.cc:548] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.905696 39738 consensus_queue.cc:176] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.905758 39768 tablet_bootstrap.cc:541] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.905846 39768 tablet_bootstrap.cc:484] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.905879 39743 tablet_bootstrap.cc:382] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.906016 39743 tablet_bootstrap.cc:541] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.906141 39743 tablet_bootstrap.cc:484] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.906179 39733 raft_consensus.cc:288] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.906211 39733 raft_consensus.cc:319] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.906219 39738 ts_tablet_manager.cc:755] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s I1031 23:05:33.906224 39733 raft_consensus.cc:548] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.906252 39738 ts_tablet_manager.cc:723] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.906255 39733 consensus_queue.cc:176] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.906366 10770 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.906394 10770 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 8182189275034354a3c350318114c7e4 I1031 23:05:33.906417 10770 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(8182189275034354a3c350318114c7e4) I1031 23:05:33.906436 10770 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(8182189275034354a3c350318114c7e4) I1031 23:05:33.906453 10770 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(8182189275034354a3c350318114c7e4) I1031 23:05:33.906471 10770 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(8182189275034354a3c350318114c7e4) I1031 23:05:33.906489 10770 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(8182189275034354a3c350318114c7e4) I1031 23:05:33.906505 39738 tablet_bootstrap.cc:382] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.906507 10770 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(8182189275034354a3c350318114c7e4) I1031 23:05:33.906538 10770 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(8182189275034354a3c350318114c7e4) I1031 23:05:33.906558 10770 raft_consensus.cc:1569] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.906577 10770 raft_consensus.cc:1585] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.906652 39738 tablet_bootstrap.cc:541] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.906759 39738 tablet_bootstrap.cc:484] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.906810 39753 raft_consensus.cc:288] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.906831 39753 raft_consensus.cc:319] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.906847 39753 raft_consensus.cc:548] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.906862 39753 consensus_queue.cc:176] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.907027 39753 ts_tablet_manager.cc:755] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s I1031 23:05:33.907053 39753 ts_tablet_manager.cc:723] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.907289 39753 tablet_bootstrap.cc:382] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.907436 39753 tablet_bootstrap.cc:541] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.907532 39753 tablet_bootstrap.cc:484] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.907716 39733 ts_tablet_manager.cc:755] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s I1031 23:05:33.907748 39733 ts_tablet_manager.cc:723] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.907749 39758 raft_consensus.cc:288] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } I1031 23:05:33.907778 39758 raft_consensus.cc:319] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.907788 39758 raft_consensus.cc:548] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.907804 39758 consensus_queue.cc:176] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.907977 39758 ts_tablet_manager.cc:755] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s I1031 23:05:33.908012 39758 ts_tablet_manager.cc:723] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.908044 10754 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.908067 10754 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: f49a4f915ac048b096fcb5f4878f436d I1031 23:05:33.908084 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(f49a4f915ac048b096fcb5f4878f436d) I1031 23:05:33.908092 39733 tablet_bootstrap.cc:382] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.908103 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(f49a4f915ac048b096fcb5f4878f436d) I1031 23:05:33.908119 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(f49a4f915ac048b096fcb5f4878f436d) I1031 23:05:33.908133 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(f49a4f915ac048b096fcb5f4878f436d) I1031 23:05:33.908146 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(f49a4f915ac048b096fcb5f4878f436d) I1031 23:05:33.908157 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(f49a4f915ac048b096fcb5f4878f436d) I1031 23:05:33.908167 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(f49a4f915ac048b096fcb5f4878f436d) I1031 23:05:33.908176 10754 raft_consensus.cc:1569] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.908188 10754 raft_consensus.cc:1585] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.908236 39733 tablet_bootstrap.cc:541] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.908284 39758 tablet_bootstrap.cc:382] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.908354 39733 tablet_bootstrap.cc:484] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.908407 39758 tablet_bootstrap.cc:541] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.908515 39758 tablet_bootstrap.cc:484] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.910347 10770 ts_tablet_manager.cc:1020] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.911424 10754 ts_tablet_manager.cc:1020] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.926786 39748 tablet_bootstrap.cc:382] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.926844 39748 ts_tablet_manager.cc:728] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.027s user 0.000s sys 0.001s I1031 23:05:33.926970 39768 tablet_bootstrap.cc:382] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.927007 39768 ts_tablet_manager.cc:728] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s I1031 23:05:33.927099 39763 tablet_bootstrap.cc:382] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.927141 39763 ts_tablet_manager.cc:728] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s I1031 23:05:33.928606 39738 tablet_bootstrap.cc:382] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.928629 39743 tablet_bootstrap.cc:382] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.928666 39738 ts_tablet_manager.cc:728] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.002s I1031 23:05:33.928678 39728 tablet_bootstrap.cc:382] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.928679 39743 ts_tablet_manager.cc:728] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.001s I1031 23:05:33.928732 39728 ts_tablet_manager.cc:728] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.029s user 0.001s sys 0.001s I1031 23:05:33.928756 10782 raft_consensus.cc:2206] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:33.929798 39758 tablet_bootstrap.cc:382] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.929834 39758 ts_tablet_manager.cc:728] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.000s I1031 23:05:33.929958 39748 raft_consensus.cc:288] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:33.929991 39748 raft_consensus.cc:319] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.929991 39733 tablet_bootstrap.cc:382] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.930003 39748 raft_consensus.cc:548] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.930033 39748 consensus_queue.cc:176] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.930037 39733 ts_tablet_manager.cc:728] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.022s user 0.001s sys 0.001s I1031 23:05:33.930107 39763 raft_consensus.cc:288] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f53a0d3ef7e44ebb0365c800752d5bd" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st01.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.930138 39763 raft_consensus.cc:319] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.930155 39763 raft_consensus.cc:548] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.930173 39763 consensus_queue.cc:176] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.930239 39768 raft_consensus.cc:288] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0f53a0d3ef7e44ebb0365c800752d5bd" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st01.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.930259 39768 raft_consensus.cc:319] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.930263 39753 tablet_bootstrap.cc:382] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.930268 39768 raft_consensus.cc:548] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.930295 39753 ts_tablet_manager.cc:728] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.023s user 0.001s sys 0.000s I1031 23:05:33.930297 39768 consensus_queue.cc:176] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.931008 39743 raft_consensus.cc:288] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.931031 39743 raft_consensus.cc:319] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.931044 39743 raft_consensus.cc:548] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.931057 39743 consensus_queue.cc:176] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.931107 39738 raft_consensus.cc:288] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } I1031 23:05:33.931129 39738 raft_consensus.cc:319] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.931133 39728 raft_consensus.cc:288] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:05:33.931139 39738 raft_consensus.cc:548] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.931159 39728 raft_consensus.cc:319] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.931165 39738 consensus_queue.cc:176] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.931174 39728 raft_consensus.cc:548] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.931190 39728 consensus_queue.cc:176] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.932554 39758 raft_consensus.cc:288] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.932577 39758 raft_consensus.cc:319] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.932587 39758 raft_consensus.cc:548] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.932601 39758 consensus_queue.cc:176] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.932652 39733 raft_consensus.cc:288] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:33.932674 39733 raft_consensus.cc:319] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.932687 39733 raft_consensus.cc:548] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.932699 39733 consensus_queue.cc:176] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.932813 39763 ts_tablet_manager.cc:755] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.006s user 0.001s sys 0.002s I1031 23:05:33.932914 39763 ts_tablet_manager.cc:723] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.932932 39753 raft_consensus.cc:288] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0f53a0d3ef7e44ebb0365c800752d5bd" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st01.bj" port: 18700 } } I1031 23:05:33.932960 39753 raft_consensus.cc:319] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.932976 39753 raft_consensus.cc:548] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.932996 39748 ts_tablet_manager.cc:755] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.006s user 0.001s sys 0.001s I1031 23:05:33.932996 39753 consensus_queue.cc:176] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.933030 39748 ts_tablet_manager.cc:723] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.933027 39768 ts_tablet_manager.cc:755] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.006s user 0.002s sys 0.001s I1031 23:05:33.933075 39768 ts_tablet_manager.cc:723] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.933408 39748 tablet_bootstrap.cc:382] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.933434 39763 tablet_bootstrap.cc:382] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.933562 39768 tablet_bootstrap.cc:382] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.933598 39748 tablet_bootstrap.cc:541] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.933601 39763 tablet_bootstrap.cc:541] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.933712 39768 tablet_bootstrap.cc:541] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.933779 39748 tablet_bootstrap.cc:484] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.933859 39763 tablet_bootstrap.cc:484] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.933902 39768 tablet_bootstrap.cc:484] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.933959 39738 ts_tablet_manager.cc:755] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.000s sys 0.000s I1031 23:05:33.934002 39738 ts_tablet_manager.cc:723] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.934036 39743 ts_tablet_manager.cc:755] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s I1031 23:05:33.934064 39728 ts_tablet_manager.cc:755] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s I1031 23:05:33.934073 39743 ts_tablet_manager.cc:723] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.934094 39728 ts_tablet_manager.cc:723] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.934424 39743 tablet_bootstrap.cc:382] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.934439 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.934479 39728 tablet_bootstrap.cc:382] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.934538 10767 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 6a813ca355f749b6b31a773aa0f2ac7a I1031 23:05:33.934543 39738 tablet_bootstrap.cc:382] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.934545 39743 tablet_bootstrap.cc:541] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.934562 10767 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(6a813ca355f749b6b31a773aa0f2ac7a) I1031 23:05:33.934572 10767 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(6a813ca355f749b6b31a773aa0f2ac7a) I1031 23:05:33.934582 10767 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(6a813ca355f749b6b31a773aa0f2ac7a) I1031 23:05:33.934595 10767 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(6a813ca355f749b6b31a773aa0f2ac7a) I1031 23:05:33.934609 10767 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(6a813ca355f749b6b31a773aa0f2ac7a) I1031 23:05:33.934639 10767 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(6a813ca355f749b6b31a773aa0f2ac7a) I1031 23:05:33.934643 39728 tablet_bootstrap.cc:541] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.934674 39743 tablet_bootstrap.cc:484] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.934680 10767 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(6a813ca355f749b6b31a773aa0f2ac7a) I1031 23:05:33.934700 10767 raft_consensus.cc:1569] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.934720 10767 raft_consensus.cc:1585] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.934722 39753 ts_tablet_manager.cc:755] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s I1031 23:05:33.934756 39753 ts_tablet_manager.cc:723] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.934790 39728 tablet_bootstrap.cc:484] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.934795 39738 tablet_bootstrap.cc:541] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.934809 39733 ts_tablet_manager.cc:755] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s I1031 23:05:33.934825 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.934851 10764 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.934876 39758 ts_tablet_manager.cc:755] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s I1031 23:05:33.934882 10762 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.934988 39738 tablet_bootstrap.cc:484] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.935005 10769 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 45e2cfbf12734183aa63ce7e9e78c541 I1031 23:05:33.935029 10769 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(45e2cfbf12734183aa63ce7e9e78c541) I1031 23:05:33.935035 10764 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: bd6a9e583cad413ca558178148ac7f85 I1031 23:05:33.935042 10769 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(45e2cfbf12734183aa63ce7e9e78c541) I1031 23:05:33.935058 39758 ts_tablet_manager.cc:723] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.935024 10762 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 69b4b5f037014f10bfab077f3f98ca49 I1031 23:05:33.935077 10764 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(bd6a9e583cad413ca558178148ac7f85) I1031 23:05:33.935055 39733 ts_tablet_manager.cc:723] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.935082 10769 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(45e2cfbf12734183aa63ce7e9e78c541) I1031 23:05:33.935097 10762 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(69b4b5f037014f10bfab077f3f98ca49) I1031 23:05:33.935132 10769 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(45e2cfbf12734183aa63ce7e9e78c541) I1031 23:05:33.935134 10762 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(69b4b5f037014f10bfab077f3f98ca49) I1031 23:05:33.935153 10769 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(45e2cfbf12734183aa63ce7e9e78c541) I1031 23:05:33.935099 10764 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(bd6a9e583cad413ca558178148ac7f85) I1031 23:05:33.935164 10762 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(69b4b5f037014f10bfab077f3f98ca49) I1031 23:05:33.935170 10769 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(45e2cfbf12734183aa63ce7e9e78c541) I1031 23:05:33.935184 10764 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(bd6a9e583cad413ca558178148ac7f85) I1031 23:05:33.935199 10762 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(69b4b5f037014f10bfab077f3f98ca49) I1031 23:05:33.935191 10769 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(45e2cfbf12734183aa63ce7e9e78c541) I1031 23:05:33.935219 10764 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(bd6a9e583cad413ca558178148ac7f85) I1031 23:05:33.935223 10769 raft_consensus.cc:1569] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.935235 39753 tablet_bootstrap.cc:382] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.935223 10762 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(69b4b5f037014f10bfab077f3f98ca49) I1031 23:05:33.935251 10764 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(bd6a9e583cad413ca558178148ac7f85) I1031 23:05:33.935256 10769 raft_consensus.cc:1585] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.935271 10764 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(bd6a9e583cad413ca558178148ac7f85) I1031 23:05:33.935261 10762 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(69b4b5f037014f10bfab077f3f98ca49) I1031 23:05:33.935282 10764 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(bd6a9e583cad413ca558178148ac7f85) I1031 23:05:33.935286 10762 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(69b4b5f037014f10bfab077f3f98ca49) I1031 23:05:33.935307 10762 raft_consensus.cc:1569] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.935303 10764 raft_consensus.cc:1569] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.935330 10762 raft_consensus.cc:1585] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.935343 10764 raft_consensus.cc:1585] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.935398 39753 tablet_bootstrap.cc:541] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.935428 39758 tablet_bootstrap.cc:382] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.935495 39733 tablet_bootstrap.cc:382] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.935501 39753 tablet_bootstrap.cc:484] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.935580 39758 tablet_bootstrap.cc:541] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.935624 39733 tablet_bootstrap.cc:541] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.935684 39758 tablet_bootstrap.cc:484] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.935745 39733 tablet_bootstrap.cc:484] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.936328 39768 tablet_bootstrap.cc:382] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.936360 39768 ts_tablet_manager.cc:728] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s I1031 23:05:33.938014 39733 tablet_bootstrap.cc:382] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.938024 39763 tablet_bootstrap.cc:382] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.938055 39733 ts_tablet_manager.cc:728] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s I1031 23:05:33.938076 39758 tablet_bootstrap.cc:382] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.938077 39763 ts_tablet_manager.cc:728] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.001s I1031 23:05:33.938117 39758 ts_tablet_manager.cc:728] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s I1031 23:05:33.938138 39768 raft_consensus.cc:288] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } I1031 23:05:33.938169 39768 raft_consensus.cc:319] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.938190 39768 raft_consensus.cc:548] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.938215 39768 consensus_queue.cc:176] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.938387 39748 tablet_bootstrap.cc:382] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.938436 39748 ts_tablet_manager.cc:728] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s I1031 23:05:33.938511 39768 ts_tablet_manager.cc:755] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s I1031 23:05:33.938563 39743 tablet_bootstrap.cc:382] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.938594 39743 ts_tablet_manager.cc:728] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s I1031 23:05:33.938699 39758 raft_consensus.cc:288] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.938727 39758 raft_consensus.cc:319] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.938740 39758 raft_consensus.cc:548] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.938755 39758 consensus_queue.cc:176] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.939075 39768 ts_tablet_manager.cc:723] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.939322 10763 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.939348 10763 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 4b81a7f36c9a4b24a3294e39fdf19d9d I1031 23:05:33.939371 10763 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(4b81a7f36c9a4b24a3294e39fdf19d9d) I1031 23:05:33.939399 10763 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(4b81a7f36c9a4b24a3294e39fdf19d9d) I1031 23:05:33.939402 39768 tablet_bootstrap.cc:382] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.939419 10763 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(4b81a7f36c9a4b24a3294e39fdf19d9d) I1031 23:05:33.939435 10763 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(4b81a7f36c9a4b24a3294e39fdf19d9d) I1031 23:05:33.939453 10763 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(4b81a7f36c9a4b24a3294e39fdf19d9d) I1031 23:05:33.939469 10763 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(4b81a7f36c9a4b24a3294e39fdf19d9d) I1031 23:05:33.939488 10763 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(4b81a7f36c9a4b24a3294e39fdf19d9d) I1031 23:05:33.939502 10763 raft_consensus.cc:1569] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.939517 10763 raft_consensus.cc:1585] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.939541 39768 tablet_bootstrap.cc:541] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.939707 39768 tablet_bootstrap.cc:484] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.941223 10767 ts_tablet_manager.cc:1020] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.941519 10764 ts_tablet_manager.cc:1020] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.941606 10769 ts_tablet_manager.cc:1020] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.941669 10762 ts_tablet_manager.cc:1020] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.942380 39738 tablet_bootstrap.cc:382] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.942415 39738 ts_tablet_manager.cc:728] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.001s I1031 23:05:33.942505 39728 tablet_bootstrap.cc:382] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.942512 39758 ts_tablet_manager.cc:755] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s I1031 23:05:33.942534 39728 ts_tablet_manager.cc:728] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.002s I1031 23:05:33.942987 39758 ts_tablet_manager.cc:723] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.943282 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.943286 39748 raft_consensus.cc:288] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.943302 10759 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 802f72af09384e95acd389f46a1d9c84 I1031 23:05:33.943307 39758 tablet_bootstrap.cc:382] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.943307 39733 raft_consensus.cc:288] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } I1031 23:05:33.943310 39748 raft_consensus.cc:319] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.943321 10759 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(802f72af09384e95acd389f46a1d9c84) I1031 23:05:33.943343 39733 raft_consensus.cc:319] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.943346 39748 raft_consensus.cc:548] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.943353 10759 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(802f72af09384e95acd389f46a1d9c84) I1031 23:05:33.943364 39733 raft_consensus.cc:548] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.943377 39748 consensus_queue.cc:176] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.943382 10759 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(802f72af09384e95acd389f46a1d9c84) I1031 23:05:33.943384 39733 consensus_queue.cc:176] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.943398 10759 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(802f72af09384e95acd389f46a1d9c84) I1031 23:05:33.943416 10759 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(802f72af09384e95acd389f46a1d9c84) I1031 23:05:33.943434 10759 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(802f72af09384e95acd389f46a1d9c84) I1031 23:05:33.943446 10759 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(802f72af09384e95acd389f46a1d9c84) I1031 23:05:33.943452 39743 raft_consensus.cc:288] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:05:33.943464 39763 raft_consensus.cc:288] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:05:33.943457 10759 raft_consensus.cc:1569] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.943481 39763 raft_consensus.cc:319] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.943483 10759 raft_consensus.cc:1585] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.943497 39763 raft_consensus.cc:548] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.943500 39728 raft_consensus.cc:288] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.943517 39763 consensus_queue.cc:176] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.943523 39728 raft_consensus.cc:319] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.943532 39728 raft_consensus.cc:548] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.943547 39728 consensus_queue.cc:176] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.943482 39743 raft_consensus.cc:319] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.943583 39733 ts_tablet_manager.cc:755] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.000s I1031 23:05:33.943596 39753 tablet_bootstrap.cc:382] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.943604 39743 raft_consensus.cc:548] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.943626 39733 ts_tablet_manager.cc:723] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.943634 39743 consensus_queue.cc:176] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.943639 39753 ts_tablet_manager.cc:728] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.001s I1031 23:05:33.943672 39748 ts_tablet_manager.cc:755] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s I1031 23:05:33.943692 39758 tablet_bootstrap.cc:541] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.943711 39748 ts_tablet_manager.cc:723] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.943732 39738 raft_consensus.cc:288] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.943750 39738 raft_consensus.cc:319] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.943759 39738 raft_consensus.cc:548] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.943773 39738 consensus_queue.cc:176] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.943826 10758 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.943832 39758 tablet_bootstrap.cc:484] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.944003 39768 tablet_bootstrap.cc:382] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.944033 39768 ts_tablet_manager.cc:728] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s I1031 23:05:33.944042 39733 tablet_bootstrap.cc:382] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.944097 39748 tablet_bootstrap.cc:382] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.944169 10758 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 582c1281bf5147b2bd40c9a950a1152f I1031 23:05:33.944200 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(582c1281bf5147b2bd40c9a950a1152f) I1031 23:05:33.944222 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(582c1281bf5147b2bd40c9a950a1152f) I1031 23:05:33.944231 39748 tablet_bootstrap.cc:541] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.944223 39733 tablet_bootstrap.cc:541] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.944247 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(582c1281bf5147b2bd40c9a950a1152f) I1031 23:05:33.944255 39753 raft_consensus.cc:288] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:33.944268 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(582c1281bf5147b2bd40c9a950a1152f) I1031 23:05:33.944283 39753 raft_consensus.cc:319] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.944285 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(582c1281bf5147b2bd40c9a950a1152f) I1031 23:05:33.944299 39753 raft_consensus.cc:548] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.944308 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(582c1281bf5147b2bd40c9a950a1152f) I1031 23:05:33.944321 39753 consensus_queue.cc:176] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.944327 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(582c1281bf5147b2bd40c9a950a1152f) I1031 23:05:33.944345 10758 raft_consensus.cc:1569] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.944361 10758 raft_consensus.cc:1585] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.944388 10756 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.944393 39733 tablet_bootstrap.cc:484] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.944409 10756 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 388697c0c21a4b158a92f2b292b8012f I1031 23:05:33.944396 10763 ts_tablet_manager.cc:1020] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.944427 10756 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(388697c0c21a4b158a92f2b292b8012f) I1031 23:05:33.944427 39748 tablet_bootstrap.cc:484] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.944447 10756 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(388697c0c21a4b158a92f2b292b8012f) I1031 23:05:33.944473 10756 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(388697c0c21a4b158a92f2b292b8012f) I1031 23:05:33.944488 10756 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(388697c0c21a4b158a92f2b292b8012f) I1031 23:05:33.944500 10756 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(388697c0c21a4b158a92f2b292b8012f) I1031 23:05:33.944514 10756 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(388697c0c21a4b158a92f2b292b8012f) I1031 23:05:33.944525 10756 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(388697c0c21a4b158a92f2b292b8012f) I1031 23:05:33.944533 10756 raft_consensus.cc:1569] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.944545 10756 raft_consensus.cc:1585] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.946357 39728 ts_tablet_manager.cc:755] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s I1031 23:05:33.946390 39728 ts_tablet_manager.cc:723] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.946610 10773 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.946641 39728 tablet_bootstrap.cc:382] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.946641 10773 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: b758ec3274c84dcbb110cf5aa3dca9aa I1031 23:05:33.946683 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(b758ec3274c84dcbb110cf5aa3dca9aa) I1031 23:05:33.946701 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(b758ec3274c84dcbb110cf5aa3dca9aa) I1031 23:05:33.946717 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(b758ec3274c84dcbb110cf5aa3dca9aa) I1031 23:05:33.946736 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(b758ec3274c84dcbb110cf5aa3dca9aa) I1031 23:05:33.946756 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(b758ec3274c84dcbb110cf5aa3dca9aa) I1031 23:05:33.946769 39728 tablet_bootstrap.cc:541] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.946772 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(b758ec3274c84dcbb110cf5aa3dca9aa) I1031 23:05:33.946804 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(b758ec3274c84dcbb110cf5aa3dca9aa) I1031 23:05:33.946816 10773 raft_consensus.cc:1569] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.946835 10768 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.946836 10773 raft_consensus.cc:1585] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.946884 39728 tablet_bootstrap.cc:484] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.946902 39738 ts_tablet_manager.cc:755] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.000s sys 0.001s I1031 23:05:33.946990 39738 ts_tablet_manager.cc:723] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.947027 39763 ts_tablet_manager.cc:755] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s I1031 23:05:33.947064 39763 ts_tablet_manager.cc:723] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.947080 39743 ts_tablet_manager.cc:755] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.008s user 0.000s sys 0.002s I1031 23:05:33.947113 39743 ts_tablet_manager.cc:723] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.947168 10768 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.947186 10768 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 8ed42ada2d6846f3b3b70c9d9c13c986 I1031 23:05:33.947203 10768 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(8ed42ada2d6846f3b3b70c9d9c13c986) I1031 23:05:33.947218 10768 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(8ed42ada2d6846f3b3b70c9d9c13c986) I1031 23:05:33.947237 10768 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(8ed42ada2d6846f3b3b70c9d9c13c986) I1031 23:05:33.947258 10768 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(8ed42ada2d6846f3b3b70c9d9c13c986) I1031 23:05:33.947278 10768 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(8ed42ada2d6846f3b3b70c9d9c13c986) I1031 23:05:33.947291 10768 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(8ed42ada2d6846f3b3b70c9d9c13c986) I1031 23:05:33.947306 10768 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(8ed42ada2d6846f3b3b70c9d9c13c986) I1031 23:05:33.947314 39738 tablet_bootstrap.cc:382] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.947325 10768 raft_consensus.cc:1569] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.947350 10768 raft_consensus.cc:1585] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.947391 39763 tablet_bootstrap.cc:382] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.947413 39743 tablet_bootstrap.cc:382] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.947445 39738 tablet_bootstrap.cc:541] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.947522 39763 tablet_bootstrap.cc:541] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.947540 39743 tablet_bootstrap.cc:541] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.947577 10771 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.947585 39738 tablet_bootstrap.cc:484] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.947608 10771 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 812b248175004ae8992ff2660fc9d520 I1031 23:05:33.947625 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(812b248175004ae8992ff2660fc9d520) I1031 23:05:33.947640 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(812b248175004ae8992ff2660fc9d520) I1031 23:05:33.947644 39763 tablet_bootstrap.cc:484] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.947654 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(812b248175004ae8992ff2660fc9d520) I1031 23:05:33.947662 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(812b248175004ae8992ff2660fc9d520) I1031 23:05:33.947671 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(812b248175004ae8992ff2660fc9d520) I1031 23:05:33.947682 39743 tablet_bootstrap.cc:484] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.947684 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(812b248175004ae8992ff2660fc9d520) I1031 23:05:33.947713 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(812b248175004ae8992ff2660fc9d520) I1031 23:05:33.947726 10771 raft_consensus.cc:1569] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.947739 10771 raft_consensus.cc:1585] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.947871 10759 ts_tablet_manager.cc:1020] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.947950 10772 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.948601 39768 raft_consensus.cc:288] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0f53a0d3ef7e44ebb0365c800752d5bd" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st01.bj" port: 18700 } } I1031 23:05:33.948629 39768 raft_consensus.cc:319] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.948642 39768 raft_consensus.cc:548] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.948662 39768 consensus_queue.cc:176] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.948681 39753 ts_tablet_manager.cc:755] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.001s sys 0.001s I1031 23:05:33.948715 39753 ts_tablet_manager.cc:723] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.948890 10772 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.948912 10772 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 46fc46e028434bc69f467922a9a59101 I1031 23:05:33.948928 10772 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(46fc46e028434bc69f467922a9a59101) I1031 23:05:33.948949 10772 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(46fc46e028434bc69f467922a9a59101) I1031 23:05:33.948962 10772 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(46fc46e028434bc69f467922a9a59101) I1031 23:05:33.948976 10772 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(46fc46e028434bc69f467922a9a59101) I1031 23:05:33.948988 10772 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(46fc46e028434bc69f467922a9a59101) I1031 23:05:33.949000 10772 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(46fc46e028434bc69f467922a9a59101) I1031 23:05:33.949012 10772 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(46fc46e028434bc69f467922a9a59101) I1031 23:05:33.949023 10772 raft_consensus.cc:1569] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.949035 10772 raft_consensus.cc:1585] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.949129 39753 tablet_bootstrap.cc:382] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.949245 39753 tablet_bootstrap.cc:541] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.949360 39753 tablet_bootstrap.cc:484] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.949399 39728 tablet_bootstrap.cc:382] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.949425 39728 ts_tablet_manager.cc:728] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s I1031 23:05:33.949568 10791 raft_consensus.cc:2206] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:33.949657 39743 tablet_bootstrap.cc:382] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.949686 39743 ts_tablet_manager.cc:728] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s I1031 23:05:33.949780 39748 tablet_bootstrap.cc:382] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.949784 39763 tablet_bootstrap.cc:382] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.949816 39748 ts_tablet_manager.cc:728] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s I1031 23:05:33.949818 39763 ts_tablet_manager.cc:728] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s I1031 23:05:33.949895 39733 tablet_bootstrap.cc:382] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.949904 39758 tablet_bootstrap.cc:382] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.949914 39768 ts_tablet_manager.cc:755] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s I1031 23:05:33.949939 39733 ts_tablet_manager.cc:728] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s I1031 23:05:33.949945 39738 tablet_bootstrap.cc:382] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.949949 39758 ts_tablet_manager.cc:728] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s I1031 23:05:33.949977 39738 ts_tablet_manager.cc:728] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s I1031 23:05:33.950027 39753 tablet_bootstrap.cc:382] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.950058 39753 ts_tablet_manager.cc:728] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s I1031 23:05:33.950074 39728 raft_consensus.cc:288] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.950103 39728 raft_consensus.cc:319] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.950120 39728 raft_consensus.cc:548] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.950142 39728 consensus_queue.cc:176] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.950340 39728 ts_tablet_manager.cc:755] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s I1031 23:05:33.950389 39743 raft_consensus.cc:288] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } I1031 23:05:33.950414 39743 raft_consensus.cc:319] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.950430 39743 raft_consensus.cc:548] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.950453 39743 consensus_queue.cc:176] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.950538 39768 ts_tablet_manager.cc:723] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.950551 39728 ts_tablet_manager.cc:723] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.950742 10760 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.950826 10760 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 117cd74bcdab4942901acb0005cdd702 I1031 23:05:33.950841 39728 tablet_bootstrap.cc:382] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.950845 10760 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(117cd74bcdab4942901acb0005cdd702) I1031 23:05:33.950863 39768 tablet_bootstrap.cc:382] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.950875 10760 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(117cd74bcdab4942901acb0005cdd702) I1031 23:05:33.950889 10760 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(117cd74bcdab4942901acb0005cdd702) I1031 23:05:33.950902 10760 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(117cd74bcdab4942901acb0005cdd702) I1031 23:05:33.950915 10760 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(117cd74bcdab4942901acb0005cdd702) I1031 23:05:33.950925 10760 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(117cd74bcdab4942901acb0005cdd702) I1031 23:05:33.950937 10760 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(117cd74bcdab4942901acb0005cdd702) I1031 23:05:33.950950 10760 raft_consensus.cc:1569] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.950964 10760 raft_consensus.cc:1585] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.950964 10766 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.950985 39728 tablet_bootstrap.cc:541] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.950994 10766 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: de37a68992504da8a4640f8ab4653544 I1031 23:05:33.950997 39768 tablet_bootstrap.cc:541] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.951017 10766 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(de37a68992504da8a4640f8ab4653544) I1031 23:05:33.951035 10766 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(de37a68992504da8a4640f8ab4653544) I1031 23:05:33.951048 10766 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(de37a68992504da8a4640f8ab4653544) I1031 23:05:33.951062 10766 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(de37a68992504da8a4640f8ab4653544) I1031 23:05:33.951079 10766 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(de37a68992504da8a4640f8ab4653544) I1031 23:05:33.951086 39728 tablet_bootstrap.cc:484] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.951093 10766 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(de37a68992504da8a4640f8ab4653544) I1031 23:05:33.951114 10766 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(de37a68992504da8a4640f8ab4653544) I1031 23:05:33.951125 10766 raft_consensus.cc:1569] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.951136 10766 raft_consensus.cc:1585] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.951141 39768 tablet_bootstrap.cc:484] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.951628 10773 ts_tablet_manager.cc:1020] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.952512 10771 ts_tablet_manager.cc:1020] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.952651 10756 ts_tablet_manager.cc:1020] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.953626 39763 raft_consensus.cc:288] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } I1031 23:05:33.953649 39763 raft_consensus.cc:319] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.953654 39748 raft_consensus.cc:288] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:05:33.953660 39763 raft_consensus.cc:548] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.953685 39748 raft_consensus.cc:319] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.953694 39748 raft_consensus.cc:548] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.953706 39763 consensus_queue.cc:176] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.953706 39748 consensus_queue.cc:176] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.953748 39758 raft_consensus.cc:288] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.953773 39758 raft_consensus.cc:319] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.953789 39758 raft_consensus.cc:548] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.953809 39758 consensus_queue.cc:176] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.953832 39738 raft_consensus.cc:288] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.953837 39753 raft_consensus.cc:288] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } I1031 23:05:33.953872 39738 raft_consensus.cc:319] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.953886 39753 raft_consensus.cc:319] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.953889 39738 raft_consensus.cc:548] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.953905 39753 raft_consensus.cc:548] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.953910 39738 consensus_queue.cc:176] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.953914 10772 ts_tablet_manager.cc:1020] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.953917 39743 ts_tablet_manager.cc:755] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s I1031 23:05:33.953936 39753 consensus_queue.cc:176] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.953960 39743 ts_tablet_manager.cc:723] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.953966 39733 raft_consensus.cc:288] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.953989 39748 ts_tablet_manager.cc:755] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s I1031 23:05:33.953990 39733 raft_consensus.cc:319] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.954012 39733 raft_consensus.cc:548] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.954020 39748 ts_tablet_manager.cc:723] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.954038 39733 consensus_queue.cc:176] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.954162 39758 ts_tablet_manager.cc:755] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.001s sys 0.001s I1031 23:05:33.954190 39758 ts_tablet_manager.cc:723] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:33.954226 39743 tablet_bootstrap.cc:382] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.954241 10755 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.954254 39763 ts_tablet_manager.cc:755] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s I1031 23:05:33.954303 39748 tablet_bootstrap.cc:382] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.954334 10755 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 0c2abca80c8f4fdaa788afd6a931df9d I1031 23:05:33.954360 39743 tablet_bootstrap.cc:541] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.954360 10755 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(0c2abca80c8f4fdaa788afd6a931df9d) I1031 23:05:33.954401 10755 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(0c2abca80c8f4fdaa788afd6a931df9d) I1031 23:05:33.954423 10755 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(0c2abca80c8f4fdaa788afd6a931df9d) I1031 23:05:33.954443 39748 tablet_bootstrap.cc:541] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.954444 10755 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(0c2abca80c8f4fdaa788afd6a931df9d) I1031 23:05:33.954471 39758 tablet_bootstrap.cc:382] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:33.954483 10755 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(0c2abca80c8f4fdaa788afd6a931df9d) I1031 23:05:33.954494 39743 tablet_bootstrap.cc:484] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.954504 10755 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(0c2abca80c8f4fdaa788afd6a931df9d) I1031 23:05:33.954520 10755 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(0c2abca80c8f4fdaa788afd6a931df9d) I1031 23:05:33.954538 10755 raft_consensus.cc:1569] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.954556 10755 raft_consensus.cc:1585] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.954560 39748 tablet_bootstrap.cc:484] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.954599 10765 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.954627 10765 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: c94f26e6a0ea4bea82140a0877c0f226 I1031 23:05:33.954628 10761 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:33.954637 39758 tablet_bootstrap.cc:541] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:33.954651 10761 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 1cebccaea10d49478c004cca39676556 I1031 23:05:33.954664 39738 ts_tablet_manager.cc:755] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s I1031 23:05:33.954670 10765 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(c94f26e6a0ea4bea82140a0877c0f226) I1031 23:05:33.954679 10761 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(1cebccaea10d49478c004cca39676556) I1031 23:05:33.954684 10765 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(c94f26e6a0ea4bea82140a0877c0f226) I1031 23:05:33.954695 10761 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(1cebccaea10d49478c004cca39676556) I1031 23:05:33.954701 10765 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(c94f26e6a0ea4bea82140a0877c0f226) I1031 23:05:33.954704 10761 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(1cebccaea10d49478c004cca39676556) I1031 23:05:33.954716 10765 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(c94f26e6a0ea4bea82140a0877c0f226) I1031 23:05:33.954722 10761 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(1cebccaea10d49478c004cca39676556) I1031 23:05:33.954733 10765 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(c94f26e6a0ea4bea82140a0877c0f226) I1031 23:05:33.954738 10761 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(1cebccaea10d49478c004cca39676556) I1031 23:05:33.954754 10761 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(1cebccaea10d49478c004cca39676556) I1031 23:05:33.954749 10765 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(c94f26e6a0ea4bea82140a0877c0f226) I1031 23:05:33.954747 39758 tablet_bootstrap.cc:484] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:33.954769 10761 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(1cebccaea10d49478c004cca39676556) I1031 23:05:33.954787 10761 raft_consensus.cc:1569] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.954790 10765 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(c94f26e6a0ea4bea82140a0877c0f226) I1031 23:05:33.954799 10761 raft_consensus.cc:1585] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.954830 10765 raft_consensus.cc:1569] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:33.954849 10765 raft_consensus.cc:1585] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:33.955313 10758 ts_tablet_manager.cc:1020] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.955999 10766 ts_tablet_manager.cc:1020] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.956079 10768 ts_tablet_manager.cc:1020] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.956543 39753 ts_tablet_manager.cc:755] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s I1031 23:05:33.956738 10760 ts_tablet_manager.cc:1020] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.956981 39768 tablet_bootstrap.cc:382] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.957006 39768 ts_tablet_manager.cc:728] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s I1031 23:05:33.957034 39733 ts_tablet_manager.cc:755] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.007s user 0.001s sys 0.000s I1031 23:05:33.957056 39728 tablet_bootstrap.cc:382] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.957098 39728 ts_tablet_manager.cc:728] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s I1031 23:05:33.957672 39728 raft_consensus.cc:288] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:05:33.957690 39768 raft_consensus.cc:288] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.957692 39728 raft_consensus.cc:319] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.957710 39768 raft_consensus.cc:319] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.957713 39728 raft_consensus.cc:548] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.957721 39768 raft_consensus.cc:548] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.957731 39728 consensus_queue.cc:176] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.957738 39768 consensus_queue.cc:176] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.957847 39743 tablet_bootstrap.cc:382] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.957880 39743 ts_tablet_manager.cc:728] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s I1031 23:05:33.957944 39768 ts_tablet_manager.cc:755] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s I1031 23:05:33.957993 39728 ts_tablet_manager.cc:755] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s I1031 23:05:33.959034 39743 raft_consensus.cc:288] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:33.959059 39743 raft_consensus.cc:319] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.959072 39743 raft_consensus.cc:548] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.959092 39743 consensus_queue.cc:176] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.959857 39743 ts_tablet_manager.cc:755] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s I1031 23:05:33.960335 10761 ts_tablet_manager.cc:1020] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.960430 10765 ts_tablet_manager.cc:1020] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.960644 10755 ts_tablet_manager.cc:1020] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:33.962713 10791 raft_consensus.cc:918] T 3125380200594db7a7b78a5539b9dfad P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 625f49b9554549e6a5c3a92b76e576cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:33.962808 10782 raft_consensus.cc:918] T 9967925d5dbd49eaadc0a1a58e7e2089 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 23f962e4a1744381ad5fa0d2d8b10241: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:33.970278 39748 tablet_bootstrap.cc:382] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.970312 39748 ts_tablet_manager.cc:728] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.001s I1031 23:05:33.970561 39758 tablet_bootstrap.cc:382] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:33.970595 39758 ts_tablet_manager.cc:728] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.001s I1031 23:05:33.970846 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(9d015a51a6ef49bd903db1f74f5872e1): real 0.080s user 0.001s sys 0.001s I1031 23:05:33.970888 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(9d015a51a6ef49bd903db1f74f5872e1) metrics: {"data dir 2.queue_time_us":202,"fdatasync":1,"fdatasync_us":36182,"tcmalloc_contention_cycles":375680,"thread_start_us":187,"threads_started":1} I1031 23:05:33.970908 39748 raft_consensus.cc:288] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:33.970924 39748 raft_consensus.cc:319] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.970934 39748 raft_consensus.cc:548] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.970949 39748 consensus_queue.cc:176] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.971642 39748 ts_tablet_manager.cc:755] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s I1031 23:05:33.972076 39758 raft_consensus.cc:288] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:33.972100 39758 raft_consensus.cc:319] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:33.972112 39758 raft_consensus.cc:548] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:33.972126 39758 consensus_queue.cc:176] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:33.972321 39758 ts_tablet_manager.cc:755] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s I1031 23:05:34.007678 40658 raft_consensus.cc:411] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.007733 40658 raft_consensus.cc:2162] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.567s I1031 23:05:34.007803 40658 raft_consensus.cc:435] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:05:34.008615 40658 leader_election.cc:216] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:05:34.008648 40658 leader_election.cc:216] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer daba39963d4346e1816089fd3a5442b5 W1031 23:05:34.008855 10718 leader_election.cc:278] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.008914 10717 leader_election.cc:348] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer daba39963d4346e1816089fd3a5442b5 I1031 23:05:34.008935 10717 leader_election.cc:243] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:05:34.008981 40661 raft_consensus.cc:2162] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.024s I1031 23:05:34.009003 40661 raft_consensus.cc:1971] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:05:34.009016 40661 raft_consensus.cc:411] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:05:34.009030 40661 raft_consensus.cc:2162] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.824s I1031 23:05:34.009042 40661 raft_consensus.cc:2206] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.009657 40661 raft_consensus.cc:435] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:05:34.010085 40661 leader_election.cc:216] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:05:34.010116 40661 leader_election.cc:216] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer daba39963d4346e1816089fd3a5442b5 W1031 23:05:34.010313 10718 leader_election.cc:278] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:34.048156 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 202 times. I1031 23:05:34.050987 10717 leader_election.cc:348] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer daba39963d4346e1816089fd3a5442b5 I1031 23:05:34.051015 10717 leader_election.cc:243] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:05:34.051090 40661 raft_consensus.cc:2162] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.528s I1031 23:05:34.051111 40661 raft_consensus.cc:1971] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:05:34.051822 40661 raft_consensus.cc:518] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:05:34.051877 40661 consensus_queue.cc:158] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } I1031 23:05:34.121672 10774 raft_consensus.cc:2206] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.121966 40572 raft_consensus.cc:411] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.121999 40572 raft_consensus.cc:2162] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.196s I1031 23:05:34.122058 40572 raft_consensus.cc:435] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0f53a0d3ef7e44ebb0365c800752d5bd" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st01.bj" port: 18700 } } I1031 23:05:34.122664 40572 leader_election.cc:216] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 I1031 23:05:34.122697 40572 leader_election.cc:216] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 0f53a0d3ef7e44ebb0365c800752d5bd I1031 23:05:34.123045 10717 leader_election.cc:362] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 0f53a0d3ef7e44ebb0365c800752d5bd. Message: Invalid argument: T 648eb376255a4dd4bb5818262efcc813 P 0f53a0d3ef7e44ebb0365c800752d5bd [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.123116 10719 leader_election.cc:362] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer d3eb30bd1bc34b95902fa61724239553. Message: Invalid argument: T 648eb376255a4dd4bb5818262efcc813 P d3eb30bd1bc34b95902fa61724239553 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.123145 10719 leader_election.cc:243] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:34.123184 40576 raft_consensus.cc:2162] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.996s I1031 23:05:34.123204 40576 raft_consensus.cc:2206] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.125732 10790 raft_consensus.cc:2206] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.180534 10778 raft_consensus.cc:2206] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.197633 10778 raft_consensus.cc:918] T 69b8c3e3568c4c19b638bb66e46a95a1 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 625f49b9554549e6a5c3a92b76e576cf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.197791 10790 raft_consensus.cc:918] T 1d674793183b4a3fb8f88dc864e9059a P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 0fc477287c154136abc1e2f3c82fb7f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.197803 40576 raft_consensus.cc:1908] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:34.197886 10774 raft_consensus.cc:918] T 9710e0187e434acfa54ca8f8504434ed P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer daba39963d4346e1816089fd3a5442b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.201951 40559 raft_consensus.cc:411] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.201985 40559 raft_consensus.cc:2162] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.639s I1031 23:05:34.202041 40559 raft_consensus.cc:435] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:34.202734 40559 leader_election.cc:216] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:34.202764 40559 leader_election.cc:216] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:05:34.203023 10719 leader_election.cc:362] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 193181be29284287b25df2ee92938f0b. Message: Invalid argument: T 090accc17fae4e60b7bddb6dd0bf2ef8 P 193181be29284287b25df2ee92938f0b [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.203078 10720 leader_election.cc:362] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 23f962e4a1744381ad5fa0d2d8b10241. Message: Invalid argument: T 090accc17fae4e60b7bddb6dd0bf2ef8 P 23f962e4a1744381ad5fa0d2d8b10241 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.203101 10720 leader_election.cc:243] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:34.203148 40571 raft_consensus.cc:2162] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.923s I1031 23:05:34.203171 40571 raft_consensus.cc:2206] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.203725 40571 raft_consensus.cc:1908] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:34.204766 40547 raft_consensus.cc:411] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.204800 40547 raft_consensus.cc:2162] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.799s I1031 23:05:34.204850 40547 raft_consensus.cc:435] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } I1031 23:05:34.205265 40547 leader_election.cc:216] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:05:34.205307 40547 leader_election.cc:216] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:05:34.205606 10720 leader_election.cc:362] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 23f962e4a1744381ad5fa0d2d8b10241. Message: Invalid argument: T 6f911a94e20d47aa9efaf9ffe3914c3b P 23f962e4a1744381ad5fa0d2d8b10241 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.205659 10718 leader_election.cc:362] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 0fc477287c154136abc1e2f3c82fb7f9. Message: Invalid argument: T 6f911a94e20d47aa9efaf9ffe3914c3b P 0fc477287c154136abc1e2f3c82fb7f9 [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.205680 10718 leader_election.cc:243] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:34.205721 40549 raft_consensus.cc:2162] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.814s I1031 23:05:34.205742 40549 raft_consensus.cc:2206] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.244460 10778 raft_consensus.cc:2206] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.298444 10778 raft_consensus.cc:918] T 865b2d5788f44598b169ca6415314610 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer daba39963d4346e1816089fd3a5442b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.298532 40549 raft_consensus.cc:1908] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:34.298595 10790 raft_consensus.cc:918] T 6f911a94e20d47aa9efaf9ffe3914c3b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 0fc477287c154136abc1e2f3c82fb7f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.334942 40645 raft_consensus.cc:411] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.334981 40645 raft_consensus.cc:2162] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.223s I1031 23:05:34.335042 40645 raft_consensus.cc:435] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:34.335866 40645 leader_election.cc:216] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:05:34.335923 40645 leader_election.cc:216] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 0fc477287c154136abc1e2f3c82fb7f9 W1031 23:05:34.336129 10720 leader_election.cc:278] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.336184 40506 raft_consensus.cc:411] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.336212 40506 raft_consensus.cc:2162] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.926s I1031 23:05:34.336244 10718 leader_election.cc:348] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:05:34.336261 10718 leader_election.cc:243] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:05:34.336277 40506 raft_consensus.cc:435] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } I1031 23:05:34.336308 40652 raft_consensus.cc:2162] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.998s I1031 23:05:34.336329 40652 raft_consensus.cc:1971] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:05:34.336341 40652 raft_consensus.cc:411] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:05:34.336361 40652 raft_consensus.cc:2162] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.175s I1031 23:05:34.336375 40652 raft_consensus.cc:2206] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.336851 40506 leader_election.cc:216] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 I1031 23:05:34.336881 40506 leader_election.cc:216] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:05:34.337149 10720 leader_election.cc:362] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 044f1da9a27c46acb82b1386f829f4dc. Message: Invalid argument: T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 044f1da9a27c46acb82b1386f829f4dc [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.337225 10719 leader_election.cc:362] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer d3eb30bd1bc34b95902fa61724239553. Message: Invalid argument: T 1c64d9f8e8c740c8ae73f0d7357fcbfd P d3eb30bd1bc34b95902fa61724239553 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.337245 10719 leader_election.cc:243] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:34.337297 40508 raft_consensus.cc:2162] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.758s I1031 23:05:34.337322 40508 raft_consensus.cc:2206] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.340144 40652 raft_consensus.cc:435] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:34.340198 40508 raft_consensus.cc:1908] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:34.340253 10778 raft_consensus.cc:918] T 1c64d9f8e8c740c8ae73f0d7357fcbfd P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 044f1da9a27c46acb82b1386f829f4dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.340692 40652 leader_election.cc:216] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:05:34.340721 40652 leader_election.cc:216] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 0fc477287c154136abc1e2f3c82fb7f9 W1031 23:05:34.340966 10720 leader_election.cc:278] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.341828 10718 leader_election.cc:348] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:05:34.341845 10718 leader_election.cc:243] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:05:34.341879 40652 raft_consensus.cc:2162] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.921s I1031 23:05:34.341897 40652 raft_consensus.cc:1971] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:05:34.342175 40652 raft_consensus.cc:518] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:05:34.342232 40652 consensus_queue.cc:158] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:34.345727 10778 raft_consensus.cc:2206] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.346345 10778 raft_consensus.cc:918] T 67cc714357a24166a1354f3d7276848f P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 0fc477287c154136abc1e2f3c82fb7f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.349045 10778 raft_consensus.cc:918] T 648eb376255a4dd4bb5818262efcc813 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 0f53a0d3ef7e44ebb0365c800752d5bd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.369911 40528 raft_consensus.cc:411] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.369971 40528 raft_consensus.cc:2162] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.916s I1031 23:05:34.370087 40528 raft_consensus.cc:435] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } I1031 23:05:34.370833 40528 leader_election.cc:216] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:05:34.370896 40528 leader_election.cc:216] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:05:34.371161 10718 leader_election.cc:362] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 625f49b9554549e6a5c3a92b76e576cf. Message: Invalid argument: T d528d43805c44cc398cfba41658e36f0 P 625f49b9554549e6a5c3a92b76e576cf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.371210 10720 leader_election.cc:362] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 044f1da9a27c46acb82b1386f829f4dc. Message: Invalid argument: T d528d43805c44cc398cfba41658e36f0 P 044f1da9a27c46acb82b1386f829f4dc [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.371240 10720 leader_election.cc:243] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:34.371291 40533 raft_consensus.cc:2162] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.056s I1031 23:05:34.371311 40533 raft_consensus.cc:2206] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.385676 40568 raft_consensus.cc:411] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.385725 40568 raft_consensus.cc:2162] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.157s I1031 23:05:34.385792 40568 raft_consensus.cc:435] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:34.386427 40568 leader_election.cc:216] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer daba39963d4346e1816089fd3a5442b5 I1031 23:05:34.386476 40568 leader_election.cc:216] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:05:34.386755 10720 leader_election.cc:362] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer 23f962e4a1744381ad5fa0d2d8b10241. Message: Invalid argument: T e53178c96c3c4ad2b3068e9674d1259d P 23f962e4a1744381ad5fa0d2d8b10241 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.386790 10717 leader_election.cc:362] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote denied by peer daba39963d4346e1816089fd3a5442b5. Message: Invalid argument: T e53178c96c3c4ad2b3068e9674d1259d P daba39963d4346e1816089fd3a5442b5 [term 1 LEADER]: Leader pre-election vote request: Denying vote to candidate 0a8bbcbbb5a6459aa5258464755db3bb for term 1 because replica is either leader or believes a valid leader to be alive. I1031 23:05:34.386821 10717 leader_election.cc:243] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:34.386865 40577 raft_consensus.cc:2162] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.709s I1031 23:05:34.386885 40577 raft_consensus.cc:2206] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.399232 40533 raft_consensus.cc:1908] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:34.399305 10778 raft_consensus.cc:918] T d528d43805c44cc398cfba41658e36f0 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 044f1da9a27c46acb82b1386f829f4dc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.399313 40577 raft_consensus.cc:1908] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:34.399363 10790 raft_consensus.cc:918] T e53178c96c3c4ad2b3068e9674d1259d P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer daba39963d4346e1816089fd3a5442b5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.422183 10778 raft_consensus.cc:918] T 090accc17fae4e60b7bddb6dd0bf2ef8 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 193181be29284287b25df2ee92938f0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) I1031 23:05:34.433851 40642 raft_consensus.cc:411] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.433884 40642 raft_consensus.cc:2162] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.597s I1031 23:05:34.433936 40642 raft_consensus.cc:435] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } I1031 23:05:34.434484 40642 leader_election.cc:216] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:34.434517 40642 leader_election.cc:216] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 0fc477287c154136abc1e2f3c82fb7f9 W1031 23:05:34.434746 10719 leader_election.cc:278] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.435057 10718 leader_election.cc:348] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:05:34.435102 10718 leader_election.cc:243] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:05:34.435183 40651 raft_consensus.cc:2162] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.072s I1031 23:05:34.435205 40651 raft_consensus.cc:1971] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:05:34.435219 40651 raft_consensus.cc:411] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:05:34.435233 40651 raft_consensus.cc:2162] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.241s I1031 23:05:34.435245 40651 raft_consensus.cc:2206] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.445164 10778 raft_consensus.cc:2162] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.927s I1031 23:05:34.445192 10778 raft_consensus.cc:1747] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0fc477287c154136abc1e2f3c82fb7f9 in term 0. I1031 23:05:34.446861 10778 raft_consensus.cc:2206] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.446888 10778 raft_consensus.cc:2162] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.615s I1031 23:05:34.450007 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.450045 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.002s I1031 23:05:34.450106 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:34.450700 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:34.450731 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:34.450944 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:34.450974 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.450999 10719 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:34.451045 40664 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.728s I1031 23:05:34.451064 40664 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:34.456691 40659 raft_consensus.cc:411] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.456718 40659 raft_consensus.cc:2162] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.699s I1031 23:05:34.456764 40659 raft_consensus.cc:435] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "625f49b9554549e6a5c3a92b76e576cf" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st10.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:34.457317 40659 leader_election.cc:216] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 625f49b9554549e6a5c3a92b76e576cf I1031 23:05:34.457346 40659 leader_election.cc:216] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b W1031 23:05:34.457589 10718 leader_election.cc:278] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 625f49b9554549e6a5c3a92b76e576cf: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:34.457618 10719 leader_election.cc:278] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.457645 10719 leader_election.cc:243] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:34.457693 40662 raft_consensus.cc:2162] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.247s I1031 23:05:34.457715 40662 raft_consensus.cc:1908] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:05:34.458142 10778 raft_consensus.cc:1747] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0fc477287c154136abc1e2f3c82fb7f9 in term 1. I1031 23:05:34.458272 40651 raft_consensus.cc:435] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } I1031 23:05:34.458784 40651 leader_election.cc:216] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:34.458812 40651 leader_election.cc:216] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 0fc477287c154136abc1e2f3c82fb7f9 W1031 23:05:34.459022 10719 leader_election.cc:278] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.459900 10718 leader_election.cc:348] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:05:34.459918 10718 leader_election.cc:243] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:05:34.459949 40651 raft_consensus.cc:2162] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.849s I1031 23:05:34.459964 40651 raft_consensus.cc:1971] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:05:34.460203 40651 raft_consensus.cc:518] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:05:34.460254 40651 consensus_queue.cc:158] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "0fc477287c154136abc1e2f3c82fb7f9" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st04.bj" port: 18700 } } I1031 23:05:34.546833 40643 raft_consensus.cc:411] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.546888 40643 raft_consensus.cc:2162] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.916s I1031 23:05:34.546967 40643 raft_consensus.cc:435] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:34.547606 40643 leader_election.cc:216] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 I1031 23:05:34.547636 40643 leader_election.cc:216] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b W1031 23:05:34.547886 10719 leader_election.cc:278] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.547942 10719 leader_election.cc:348] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer d3eb30bd1bc34b95902fa61724239553 I1031 23:05:34.547962 10719 leader_election.cc:243] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:05:34.548004 40650 raft_consensus.cc:2162] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.007s I1031 23:05:34.548022 40650 raft_consensus.cc:1971] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:05:34.548033 40650 raft_consensus.cc:411] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:05:34.548045 40650 raft_consensus.cc:2162] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.618s I1031 23:05:34.548055 40650 raft_consensus.cc:2206] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 W1031 23:05:34.548214 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 203 times. I1031 23:05:34.548851 40650 raft_consensus.cc:435] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } I1031 23:05:34.549381 40650 leader_election.cc:216] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer d3eb30bd1bc34b95902fa61724239553 I1031 23:05:34.549407 40650 leader_election.cc:216] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 193181be29284287b25df2ee92938f0b W1031 23:05:34.549640 10719 leader_election.cc:278] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.550523 10719 leader_election.cc:348] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer d3eb30bd1bc34b95902fa61724239553 I1031 23:05:34.550540 10719 leader_election.cc:243] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:05:34.550567 40650 raft_consensus.cc:2162] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.750s I1031 23:05:34.550580 40650 raft_consensus.cc:1971] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:05:34.551043 40650 raft_consensus.cc:518] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:05:34.551093 40650 consensus_queue.cc:158] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } W1031 23:05:34.552757 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 1 times. I1031 23:05:34.553434 40673 consensus_queue.cc:695] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: daba39963d4346e1816089fd3a5442b5, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:05:34.589118 40671 raft_consensus.cc:411] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.589139 40671 raft_consensus.cc:2162] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.829s I1031 23:05:34.589186 40671 raft_consensus.cc:435] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:34.589691 40671 leader_election.cc:216] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:05:34.589715 40671 leader_election.cc:216] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer daba39963d4346e1816089fd3a5442b5 W1031 23:05:34.589972 10720 leader_election.cc:278] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.590046 10717 leader_election.cc:348] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer daba39963d4346e1816089fd3a5442b5 I1031 23:05:34.590098 10717 leader_election.cc:243] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:05:34.590199 40672 raft_consensus.cc:2162] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.993s I1031 23:05:34.590221 40672 raft_consensus.cc:1971] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:05:34.590231 40672 raft_consensus.cc:411] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:05:34.590243 40672 raft_consensus.cc:2162] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.597s I1031 23:05:34.590252 40672 raft_consensus.cc:2206] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.601403 40672 raft_consensus.cc:435] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:34.601900 40672 leader_election.cc:216] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:05:34.601925 40672 leader_election.cc:216] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer daba39963d4346e1816089fd3a5442b5 W1031 23:05:34.602149 10720 leader_election.cc:278] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.602929 10717 leader_election.cc:348] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer daba39963d4346e1816089fd3a5442b5 I1031 23:05:34.602951 10717 leader_election.cc:243] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:05:34.602982 40672 raft_consensus.cc:2162] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.869s I1031 23:05:34.602994 40672 raft_consensus.cc:1971] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:05:34.603410 40672 raft_consensus.cc:518] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:05:34.603463 40672 consensus_queue.cc:158] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } peers { permanent_uuid: "daba39963d4346e1816089fd3a5442b5" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st09.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } I1031 23:05:34.665725 40639 raft_consensus.cc:411] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:34.665763 40639 raft_consensus.cc:2162] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.080s I1031 23:05:34.665833 40639 raft_consensus.cc:435] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:05:34.666605 40639 leader_election.cc:216] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:34.666637 40639 leader_election.cc:216] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer d3eb30bd1bc34b95902fa61724239553 W1031 23:05:34.666961 10719 leader_election.cc:278] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.667006 10719 leader_election.cc:348] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer d3eb30bd1bc34b95902fa61724239553 I1031 23:05:34.667016 10719 leader_election.cc:243] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:05:34.667062 40648 raft_consensus.cc:2162] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.010s I1031 23:05:34.667084 40648 raft_consensus.cc:1971] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:05:34.667100 40648 raft_consensus.cc:411] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:05:34.667111 40648 raft_consensus.cc:2162] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.651s I1031 23:05:34.667121 40648 raft_consensus.cc:2206] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:34.676530 40648 raft_consensus.cc:435] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:05:34.677057 40648 leader_election.cc:216] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:34.677091 40648 leader_election.cc:216] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer d3eb30bd1bc34b95902fa61724239553 W1031 23:05:34.677336 10719 leader_election.cc:278] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:34.678086 10719 leader_election.cc:348] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer d3eb30bd1bc34b95902fa61724239553 I1031 23:05:34.678102 10719 leader_election.cc:243] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:05:34.678139 40648 raft_consensus.cc:2162] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.141s I1031 23:05:34.678156 40648 raft_consensus.cc:1971] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:05:34.678691 40648 raft_consensus.cc:518] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:05:34.678738 40648 consensus_queue.cc:158] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "d3eb30bd1bc34b95902fa61724239553" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st05.bj" port: 18700 } } I1031 23:05:34.693372 10773 ts_tablet_manager.cc:1032] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:34.693403 10773 log.cc:876] T b758ec3274c84dcbb110cf5aa3dca9aa P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/b758ec3274c84dcbb110cf5aa3dca9aa I1031 23:05:34.693606 10773 consensus_meta.cc:71] T b758ec3274c84dcbb110cf5aa3dca9aa Deleting consensus metadata I1031 23:05:34.735491 10771 ts_tablet_manager.cc:1032] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:34.735532 10771 log.cc:876] T 812b248175004ae8992ff2660fc9d520 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/812b248175004ae8992ff2660fc9d520 I1031 23:05:34.735723 10771 consensus_meta.cc:71] T 812b248175004ae8992ff2660fc9d520 Deleting consensus metadata I1031 23:05:34.752562 10758 ts_tablet_manager.cc:1032] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:34.752602 10758 log.cc:876] T 582c1281bf5147b2bd40c9a950a1152f P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/582c1281bf5147b2bd40c9a950a1152f I1031 23:05:34.752830 10758 consensus_meta.cc:71] T 582c1281bf5147b2bd40c9a950a1152f Deleting consensus metadata W1031 23:05:34.843443 10720 consensus_peers.cc:357] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 32984b820db347e1b2d4d0786cf221f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 1 times. I1031 23:05:34.844149 40676 consensus_queue.cc:695] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 0fc477287c154136abc1e2f3c82fb7f9, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:05:34.960068 10774 raft_consensus.cc:918] T 29fdcf1d33f94660b0c77445ec7e559b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Refusing update from remote peer 0fc477287c154136abc1e2f3c82fb7f9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 1. (index mismatch) W1031 23:05:34.960870 10719 consensus_peers.cc:357] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet acb401e40e7049fbbfdbeb40474353d9. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 1 times. I1031 23:05:34.961275 40680 consensus_queue.cc:695] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 0fc477287c154136abc1e2f3c82fb7f9, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:05:35.030840 10754 ts_tablet_manager.cc:1032] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.030910 10754 log.cc:876] T f49a4f915ac048b096fcb5f4878f436d P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/f49a4f915ac048b096fcb5f4878f436d I1031 23:05:35.031219 10754 consensus_meta.cc:71] T f49a4f915ac048b096fcb5f4878f436d Deleting consensus metadata W1031 23:05:35.048336 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 204 times. I1031 23:05:35.052044 40650 consensus_queue.cc:695] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: d3eb30bd1bc34b95902fa61724239553, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false W1031 23:05:35.052469 10719 consensus_peers.cc:357] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet 721e8a57234447eabda00f5df65bd574. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 1 times. W1031 23:05:35.052829 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 2 times. W1031 23:05:35.104362 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 1 times. I1031 23:05:35.105401 40698 consensus_queue.cc:695] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: daba39963d4346e1816089fd3a5442b5, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:05:35.122925 10759 ts_tablet_manager.cc:1032] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.122977 10759 log.cc:876] T 802f72af09384e95acd389f46a1d9c84 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/802f72af09384e95acd389f46a1d9c84 I1031 23:05:35.126152 10759 consensus_meta.cc:71] T 802f72af09384e95acd389f46a1d9c84 Deleting consensus metadata I1031 23:05:35.167130 10773 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 369) took 1220ms. Request Metrics: {"fdatasync":2,"fdatasync_us":242158,"tcmalloc_contention_cycles":2816} I1031 23:05:35.167186 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.167198 10773 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 8356b43879104775b9a8c5a9e546e85b I1031 23:05:35.167217 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(8356b43879104775b9a8c5a9e546e85b) I1031 23:05:35.167245 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(8356b43879104775b9a8c5a9e546e85b) I1031 23:05:35.167258 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(8356b43879104775b9a8c5a9e546e85b) I1031 23:05:35.167284 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(8356b43879104775b9a8c5a9e546e85b) I1031 23:05:35.167309 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(8356b43879104775b9a8c5a9e546e85b) I1031 23:05:35.167320 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(8356b43879104775b9a8c5a9e546e85b) I1031 23:05:35.167333 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(8356b43879104775b9a8c5a9e546e85b) I1031 23:05:35.167348 10773 raft_consensus.cc:1569] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Raft consensus shutting down. I1031 23:05:35.167423 10773 consensus_peers.cc:381] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer d3eb30bd1bc34b95902fa61724239553 (c3-kudu-tst-st05.bj:18700): Closing peer: d3eb30bd1bc34b95902fa61724239553 I1031 23:05:35.167472 10773 consensus_peers.cc:381] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Closing peer: 193181be29284287b25df2ee92938f0b I1031 23:05:35.167490 10773 raft_consensus_state.cc:371] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Trying to abort 1 pending transactions. I1031 23:05:35.167524 10773 raft_consensus_state.cc:376] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Aborting transaction as it isn't in flight: id { term: 1 index: 1 } timestamp: 6182757722849439744 op_type: NO_OP noop_request { } I1031 23:05:35.167585 10773 raft_consensus.cc:2041] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: NO_OP replication failed: Aborted: Transaction aborted I1031 23:05:35.167596 10773 raft_consensus.cc:1585] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Raft consensus is shut down! I1031 23:05:35.170570 10773 ts_tablet_manager.cc:1020] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:35.190501 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(805311cf0a9549e08dc0767b41b859ec): real 0.944s user 0.167s sys 0.005s I1031 23:05:35.190783 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(805311cf0a9549e08dc0767b41b859ec) metrics: {"data dir 5.queue_time_us":178,"data dir 5.run_cpu_time_us":46,"data dir 5.run_wall_time_us":46,"fdatasync":1,"fdatasync_us":88570,"tcmalloc_contention_cycles":62336,"thread_start_us":167,"threads_started":1} I1031 23:05:35.207414 10757 ts_tablet_manager.cc:1032] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.207471 10757 log.cc:876] T 5110f0095bd94f67abf0aad906332efe P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/5110f0095bd94f67abf0aad906332efe I1031 23:05:35.207648 10771 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 372) took 1260ms. Request Metrics: {"fdatasync":2,"fdatasync_us":242062} I1031 23:05:35.207687 10757 consensus_meta.cc:71] T 5110f0095bd94f67abf0aad906332efe Deleting consensus metadata I1031 23:05:35.207711 10771 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.207744 10771 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 381) took 1253ms. Request Metrics: {} I1031 23:05:35.207787 10771 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.207803 10771 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 721e8a57234447eabda00f5df65bd574 I1031 23:05:35.207839 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(721e8a57234447eabda00f5df65bd574) I1031 23:05:35.207859 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(721e8a57234447eabda00f5df65bd574) I1031 23:05:35.207877 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(721e8a57234447eabda00f5df65bd574) I1031 23:05:35.207895 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(721e8a57234447eabda00f5df65bd574) I1031 23:05:35.207921 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(721e8a57234447eabda00f5df65bd574) I1031 23:05:35.207938 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(721e8a57234447eabda00f5df65bd574) I1031 23:05:35.207954 10771 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(721e8a57234447eabda00f5df65bd574) I1031 23:05:35.207983 10771 raft_consensus.cc:1569] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Raft consensus shutting down. I1031 23:05:35.208130 10771 consensus_peers.cc:381] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Closing peer: 193181be29284287b25df2ee92938f0b I1031 23:05:35.208209 10771 consensus_peers.cc:381] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer d3eb30bd1bc34b95902fa61724239553 (c3-kudu-tst-st05.bj:18700): Closing peer: d3eb30bd1bc34b95902fa61724239553 I1031 23:05:35.208299 10771 raft_consensus.cc:1585] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Raft consensus is shut down! I1031 23:05:35.211489 10771 ts_tablet_manager.cc:1020] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:35.232838 10762 ts_tablet_manager.cc:1032] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.232877 10762 log.cc:876] T 69b4b5f037014f10bfab077f3f98ca49 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/69b4b5f037014f10bfab077f3f98ca49 I1031 23:05:35.233011 10762 consensus_meta.cc:71] T 69b4b5f037014f10bfab077f3f98ca49 Deleting consensus metadata I1031 23:05:35.258321 10758 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 367) took 1314ms. Request Metrics: {"fdatasync":2,"fdatasync_us":251086,"tcmalloc_contention_cycles":18304} I1031 23:05:35.258364 10758 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.258376 10758 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: acb401e40e7049fbbfdbeb40474353d9 I1031 23:05:35.317229 10764 ts_tablet_manager.cc:1032] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.317273 10764 log.cc:876] T bd6a9e583cad413ca558178148ac7f85 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/bd6a9e583cad413ca558178148ac7f85 I1031 23:05:35.317402 10764 consensus_meta.cc:71] T bd6a9e583cad413ca558178148ac7f85 Deleting consensus metadata I1031 23:05:35.342512 10763 ts_tablet_manager.cc:1032] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.342591 10763 log.cc:876] T 4b81a7f36c9a4b24a3294e39fdf19d9d P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/4b81a7f36c9a4b24a3294e39fdf19d9d I1031 23:05:35.343011 10763 consensus_meta.cc:71] T 4b81a7f36c9a4b24a3294e39fdf19d9d Deleting consensus metadata W1031 23:05:35.343504 10720 consensus_peers.cc:357] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 32984b820db347e1b2d4d0786cf221f2. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 2 times. I1031 23:05:35.355330 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(acb401e40e7049fbbfdbeb40474353d9) I1031 23:05:35.355365 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(acb401e40e7049fbbfdbeb40474353d9) I1031 23:05:35.355381 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(acb401e40e7049fbbfdbeb40474353d9) I1031 23:05:35.355401 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(acb401e40e7049fbbfdbeb40474353d9) I1031 23:05:35.355428 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(acb401e40e7049fbbfdbeb40474353d9) I1031 23:05:35.355444 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(acb401e40e7049fbbfdbeb40474353d9) I1031 23:05:35.355460 10758 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(acb401e40e7049fbbfdbeb40474353d9) I1031 23:05:35.355474 10758 raft_consensus.cc:1569] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Raft consensus shutting down. I1031 23:05:35.355626 10758 consensus_peers.cc:381] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 0fc477287c154136abc1e2f3c82fb7f9 (c3-kudu-tst-st04.bj:18700): Closing peer: 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:05:35.355697 10758 consensus_peers.cc:381] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Closing peer: 193181be29284287b25df2ee92938f0b I1031 23:05:35.355754 10758 raft_consensus.cc:1585] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Raft consensus is shut down! I1031 23:05:35.358261 10758 ts_tablet_manager.cc:1020] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:35.367702 10761 ts_tablet_manager.cc:1032] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.367758 10761 log.cc:876] T 1cebccaea10d49478c004cca39676556 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/1cebccaea10d49478c004cca39676556 I1031 23:05:35.367920 10761 consensus_meta.cc:71] T 1cebccaea10d49478c004cca39676556 Deleting consensus metadata I1031 23:05:35.401443 10760 ts_tablet_manager.cc:1032] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.401492 10760 log.cc:876] T 117cd74bcdab4942901acb0005cdd702 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/117cd74bcdab4942901acb0005cdd702 I1031 23:05:35.401646 10760 consensus_meta.cc:71] T 117cd74bcdab4942901acb0005cdd702 Deleting consensus metadata I1031 23:05:35.435037 10766 ts_tablet_manager.cc:1032] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.435089 10766 log.cc:876] T de37a68992504da8a4640f8ab4653544 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/de37a68992504da8a4640f8ab4653544 I1031 23:05:35.435236 10766 consensus_meta.cc:71] T de37a68992504da8a4640f8ab4653544 Deleting consensus metadata W1031 23:05:35.443050 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 3 times. I1031 23:05:35.468732 10765 ts_tablet_manager.cc:1032] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.468770 10765 log.cc:876] T c94f26e6a0ea4bea82140a0877c0f226 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/c94f26e6a0ea4bea82140a0877c0f226 I1031 23:05:35.468906 10754 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 360) took 1560ms. Request Metrics: {"fdatasync":2,"fdatasync_us":108229} I1031 23:05:35.468916 10765 consensus_meta.cc:71] T c94f26e6a0ea4bea82140a0877c0f226 Deleting consensus metadata I1031 23:05:35.468961 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.468986 10754 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 32984b820db347e1b2d4d0786cf221f2 I1031 23:05:35.469013 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(32984b820db347e1b2d4d0786cf221f2) I1031 23:05:35.469033 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(32984b820db347e1b2d4d0786cf221f2) I1031 23:05:35.469053 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(32984b820db347e1b2d4d0786cf221f2) I1031 23:05:35.469069 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(32984b820db347e1b2d4d0786cf221f2) I1031 23:05:35.469094 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(32984b820db347e1b2d4d0786cf221f2) I1031 23:05:35.469107 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(32984b820db347e1b2d4d0786cf221f2) I1031 23:05:35.469120 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(32984b820db347e1b2d4d0786cf221f2) I1031 23:05:35.469131 10754 raft_consensus.cc:1569] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Raft consensus shutting down. I1031 23:05:35.469262 10754 consensus_peers.cc:381] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 0fc477287c154136abc1e2f3c82fb7f9 (c3-kudu-tst-st04.bj:18700): Closing peer: 0fc477287c154136abc1e2f3c82fb7f9 I1031 23:05:35.469339 10754 consensus_peers.cc:381] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Closing peer: 23f962e4a1744381ad5fa0d2d8b10241 I1031 23:05:35.469370 10754 raft_consensus.cc:1585] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Raft consensus is shut down! I1031 23:05:35.472285 10754 ts_tablet_manager.cc:1020] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:35.494046 10756 ts_tablet_manager.cc:1032] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.494115 10756 log.cc:876] T 388697c0c21a4b158a92f2b292b8012f P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/388697c0c21a4b158a92f2b292b8012f I1031 23:05:35.494352 10756 consensus_meta.cc:71] T 388697c0c21a4b158a92f2b292b8012f Deleting consensus metadata I1031 23:05:35.519278 10769 ts_tablet_manager.cc:1032] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.519333 10769 log.cc:876] T 45e2cfbf12734183aa63ce7e9e78c541 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/45e2cfbf12734183aa63ce7e9e78c541 I1031 23:05:35.519522 10757 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 358) took 1619ms. Request Metrics: {"fdatasync":2,"fdatasync_us":241679} I1031 23:05:35.519546 10769 consensus_meta.cc:71] T 45e2cfbf12734183aa63ce7e9e78c541 Deleting consensus metadata I1031 23:05:35.519574 10757 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.519601 10757 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 0def0264c0db4c438e16c22b23a0b43a I1031 23:05:35.519623 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(0def0264c0db4c438e16c22b23a0b43a) I1031 23:05:35.519640 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(0def0264c0db4c438e16c22b23a0b43a) I1031 23:05:35.519642 10762 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 364) took 1584ms. Request Metrics: {"fdatasync":2,"fdatasync_us":231909,"mutex_wait_us":28} I1031 23:05:35.519645 10759 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 366) took 1576ms. Request Metrics: {"fdatasync":2,"fdatasync_us":250105} I1031 23:05:35.519673 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(0def0264c0db4c438e16c22b23a0b43a) I1031 23:05:35.519704 10762 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.519718 10759 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.519716 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(0def0264c0db4c438e16c22b23a0b43a) I1031 23:05:35.519743 10762 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 386) took 1514ms. Request Metrics: {} I1031 23:05:35.519752 10759 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 387) took 1507ms. Request Metrics: {} I1031 23:05:35.519771 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(0def0264c0db4c438e16c22b23a0b43a) I1031 23:05:35.519783 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.519788 10762 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.519788 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(0def0264c0db4c438e16c22b23a0b43a) I1031 23:05:35.519814 10759 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 388) took 1175ms. Request Metrics: {} I1031 23:05:35.519815 10757 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(0def0264c0db4c438e16c22b23a0b43a) I1031 23:05:35.519830 10762 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 389) took 1058ms. Request Metrics: {} I1031 23:05:35.519842 10757 raft_consensus.cc:1569] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:05:35.519843 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.519855 10757 raft_consensus.cc:1585] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:05:35.519868 10762 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.519896 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.522593 10757 ts_tablet_manager.cc:1020] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:05:35.539141 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:35.548254 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 205 times. I1031 23:05:35.552503 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.561489 10770 ts_tablet_manager.cc:1032] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.561539 10770 log.cc:876] T 8182189275034354a3c350318114c7e4 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/8182189275034354a3c350318114c7e4 I1031 23:05:35.561770 10770 consensus_meta.cc:71] T 8182189275034354a3c350318114c7e4 Deleting consensus metadata I1031 23:05:35.561873 10759 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.561902 10762 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.573292 10762 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.583652 10762 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.587007 10762 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.604401 10762 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:35.607002 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 2 times. I1031 23:05:35.609787 10762 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.612004 10767 ts_tablet_manager.cc:1032] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.612051 10767 log.cc:876] T 6a813ca355f749b6b31a773aa0f2ac7a P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/6a813ca355f749b6b31a773aa0f2ac7a I1031 23:05:35.612179 10764 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 363) took 1677ms. Request Metrics: {"fdatasync":2,"fdatasync_us":234102} I1031 23:05:35.612241 10761 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 379) took 1657ms. Request Metrics: {"fdatasync":2,"fdatasync_us":150658,"mutex_wait_us":27} I1031 23:05:35.612270 10767 consensus_meta.cc:71] T 6a813ca355f749b6b31a773aa0f2ac7a Deleting consensus metadata I1031 23:05:35.612275 10763 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 365) took 1672ms. Request Metrics: {"fdatasync":2,"fdatasync_us":233921,"mutex_wait_us":21} I1031 23:05:35.612383 10760 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 375) took 1661ms. Request Metrics: {"fdatasync":2,"fdatasync_us":333847} I1031 23:05:35.626741 10760 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.645946 10755 ts_tablet_manager.cc:1032] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.646013 10755 log.cc:876] T 0c2abca80c8f4fdaa788afd6a931df9d P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/0c2abca80c8f4fdaa788afd6a931df9d I1031 23:05:35.646122 10766 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 376) took 1695ms. Request Metrics: {"fdatasync":2,"fdatasync_us":348963} I1031 23:05:35.646173 10755 consensus_meta.cc:71] T 0c2abca80c8f4fdaa788afd6a931df9d Deleting consensus metadata I1031 23:05:35.651502 10766 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.661895 10766 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.665607 10686 tablet.cc:911] T 037f0f4d2e0f4f52b42d6ef46d250ba8 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering stage 1 (old memrowset already frozen for inserts) I1031 23:05:35.665643 10686 compaction.cc:877] Selected 1 rowsets to compact: I1031 23:05:35.665658 10686 compaction.cc:880] memrowset(current size on disk: ~0 bytes) I1031 23:05:35.665674 10686 tablet.cc:913] T 037f0f4d2e0f4f52b42d6ef46d250ba8 P 0a8bbcbbb5a6459aa5258464755db3bb: Memstore in-memory size: 3751 bytes I1031 23:05:35.665701 10686 tablet.cc:1174] T 037f0f4d2e0f4f52b42d6ef46d250ba8 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6182757722158796800 or (T in {6182757722158796800})}] I1031 23:05:35.665928 10686 multi_column_writer.cc:85] Opened CFile writer for column ts[int32 NOT NULL] I1031 23:05:35.665968 10686 multi_column_writer.cc:85] Opened CFile writer for column id[int32 NOT NULL] I1031 23:05:35.671711 10765 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 378) took 1717ms. Request Metrics: {"fdatasync":2,"fdatasync_us":141860,"mutex_wait_us":24} I1031 23:05:35.671779 10756 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 368) took 1727ms. Request Metrics: {"fdatasync":2,"fdatasync_us":259058} I1031 23:05:35.671808 10769 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 362) took 1737ms. Request Metrics: {"fdatasync":2,"fdatasync_us":283533,"mutex_wait_us":1} I1031 23:05:35.680151 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.707587 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.708899 10769 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.710186 10769 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.722280 10770 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 359) took 1815ms. Request Metrics: {"fdatasync":2,"fdatasync_us":91886} I1031 23:05:35.747661 10767 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 361) took 1813ms. Request Metrics: {"fdatasync":2,"fdatasync_us":233866,"mutex_wait_us":8} I1031 23:05:35.785670 10686 tablet.cc:1259] T 037f0f4d2e0f4f52b42d6ef46d250ba8 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I1031 23:05:35.789541 10772 ts_tablet_manager.cc:1032] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.789594 10772 log.cc:876] T 46fc46e028434bc69f467922a9a59101 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/46fc46e028434bc69f467922a9a59101 I1031 23:05:35.789742 10772 consensus_meta.cc:71] T 46fc46e028434bc69f467922a9a59101 Deleting consensus metadata I1031 23:05:35.815011 10757 ts_tablet_manager.cc:1032] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.815084 10757 log.cc:876] T 0def0264c0db4c438e16c22b23a0b43a P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/0def0264c0db4c438e16c22b23a0b43a I1031 23:05:35.815102 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.815331 10757 consensus_meta.cc:71] T 0def0264c0db4c438e16c22b23a0b43a Deleting consensus metadata I1031 23:05:35.822609 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.827044 10767 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.840209 10768 ts_tablet_manager.cc:1032] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:05:35.840250 10768 log.cc:876] T 8ed42ada2d6846f3b3b70c9d9c13c986 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/8ed42ada2d6846f3b3b70c9d9c13c986 I1031 23:05:35.840397 10755 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 377) took 1886ms. Request Metrics: {"fdatasync":2,"fdatasync_us":141950} I1031 23:05:35.840530 10768 consensus_meta.cc:71] T 8ed42ada2d6846f3b3b70c9d9c13c986 Deleting consensus metadata I1031 23:05:35.853792 10755 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.861182 10755 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.885713 10755 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:35.916353 10771 ts_tablet_manager.cc:1032] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 1.1 I1031 23:05:35.916416 10771 log.cc:876] T 721e8a57234447eabda00f5df65bd574 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/721e8a57234447eabda00f5df65bd574 I1031 23:05:35.916785 10771 consensus_meta.cc:71] T 721e8a57234447eabda00f5df65bd574 Deleting consensus metadata I1031 23:05:35.941462 10758 ts_tablet_manager.cc:1032] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 1.1 I1031 23:05:35.941514 10758 log.cc:876] T acb401e40e7049fbbfdbeb40474353d9 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/acb401e40e7049fbbfdbeb40474353d9 I1031 23:05:35.941766 10758 consensus_meta.cc:71] T acb401e40e7049fbbfdbeb40474353d9 Deleting consensus metadata W1031 23:05:35.943132 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 4 times. I1031 23:05:35.966784 10773 ts_tablet_manager.cc:1032] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 1.1 I1031 23:05:35.966825 10773 log.cc:876] T 8356b43879104775b9a8c5a9e546e85b P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/8356b43879104775b9a8c5a9e546e85b I1031 23:05:35.967033 10772 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 374) took 2018ms. Request Metrics: {"fdatasync":2,"fdatasync_us":325668} I1031 23:05:35.967037 10757 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 385) took 2008ms. Request Metrics: {"fdatasync":2,"fdatasync_us":91257,"mutex_wait_us":1} I1031 23:05:35.967067 10773 consensus_meta.cc:71] T 8356b43879104775b9a8c5a9e546e85b Deleting consensus metadata I1031 23:05:35.967083 10768 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 371) took 2019ms. Request Metrics: {"fdatasync":2,"fdatasync_us":325573,"mutex_wait_us":85} I1031 23:05:35.967113 10771 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 382) took 2011ms. Request Metrics: {"fdatasync":2,"fdatasync_us":74840} W1031 23:05:35.981323 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 3 times. I1031 23:05:35.992157 10754 ts_tablet_manager.cc:1032] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 1.1 I1031 23:05:35.992209 10754 log.cc:876] T 32984b820db347e1b2d4d0786cf221f2 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/32984b820db347e1b2d4d0786cf221f2 I1031 23:05:35.992257 10758 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 383) took 2035ms. Request Metrics: {"fdatasync":2,"fdatasync_us":82944,"mutex_wait_us":96928} I1031 23:05:35.992336 10773 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 380) took 2037ms. Request Metrics: {"fdatasync":2,"fdatasync_us":83000,"mutex_wait_us":1,"tcmalloc_contention_cycles":384} I1031 23:05:35.992431 10754 consensus_meta.cc:71] T 32984b820db347e1b2d4d0786cf221f2 Deleting consensus metadata I1031 23:05:35.992573 10754 rpcz_store.cc:248] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 384) took 2035ms. Request Metrics: {"fdatasync":2,"fdatasync_us":91211} W1031 23:05:36.048497 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 206 times. I1031 23:05:36.078097 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:36.083873 10686 tablet.cc:1314] T 037f0f4d2e0f4f52b42d6ef46d250ba8 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush Phase 2: carrying over any updates which arrived during Phase 1 I1031 23:05:36.083935 10686 tablet.cc:1316] T 037f0f4d2e0f4f52b42d6ef46d250ba8 P 0a8bbcbbb5a6459aa5258464755db3bb: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6182757722158796800 or (T in {6182757722158796800})}] I1031 23:05:36.094559 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:36.095966 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:36.118755 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:36.127110 10754 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:36.186697 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:36.443099 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 5 times. W1031 23:05:36.481324 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 4 times. W1031 23:05:36.548481 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 207 times. I1031 23:05:36.631566 10686 tablet.cc:1358] T 037f0f4d2e0f4f52b42d6ef46d250ba8 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush successful on 16 rows (4331 bytes) I1031 23:05:36.631644 10686 maintenance_manager.cc:419] Time spent running FlushMRSOp(037f0f4d2e0f4f52b42d6ef46d250ba8): real 1.276s user 1.051s sys 0.016s I1031 23:05:36.631791 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: FlushMRSOp(037f0f4d2e0f4f52b42d6ef46d250ba8) metrics: {"cfile_init":1,"fdatasync":11,"fdatasync_us":168044,"lbm_read_time_us":10,"lbm_reads_lt_1ms":4,"lbm_write_time_us":174,"lbm_writes_lt_1ms":26,"tcmalloc_contention_cycles":174464} W1031 23:05:36.943331 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 6 times. W1031 23:05:36.981406 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 5 times. W1031 23:05:37.048607 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 208 times. W1031 23:05:37.443327 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 7 times. W1031 23:05:37.481408 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 6 times. W1031 23:05:37.548590 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 209 times. W1031 23:05:37.943423 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 8 times. W1031 23:05:37.981487 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 7 times. I1031 23:05:38.000365 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:38.000406 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 14.931s I1031 23:05:38.000454 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:38.001036 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:38.001071 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:38.001327 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:38.001384 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:38.001408 10719 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:38.001653 40717 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 9.335s I1031 23:05:38.001674 40717 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:38.048743 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 210 times. I1031 23:05:38.067153 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:38.067188 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.842s I1031 23:05:38.067229 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:38.067868 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:38.067905 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:38.068166 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:38.068313 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:38.068336 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:38.068677 40718 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.571s I1031 23:05:38.068708 40718 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:38.443349 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 9 times. W1031 23:05:38.481623 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 8 times. W1031 23:05:38.548851 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 211 times. W1031 23:05:38.943521 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 10 times. W1031 23:05:38.981698 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 9 times. I1031 23:05:39.030678 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 721e8a57234447eabda00f5df65bd574 with delete_type TABLET_DATA_DELETED (Replaced by 8df4d6c546194972a7c0cfb8bf652384 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:39.048882 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 212 times. I1031 23:05:39.059417 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 8356b43879104775b9a8c5a9e546e85b with delete_type TABLET_DATA_DELETED (Replaced by 29fdcf1d33f94660b0c77445ec7e559b at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:39.121393 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 1cebccaea10d49478c004cca39676556 with delete_type TABLET_DATA_DELETED (Replaced by b4e182b626554f3597d5c733c80997e6 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:39.139487 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 32984b820db347e1b2d4d0786cf221f2 with delete_type TABLET_DATA_DELETED (Replaced by eeb3a3cb417b466abffd16749d165a61 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:39.149384 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 0c2abca80c8f4fdaa788afd6a931df9d with delete_type TABLET_DATA_DELETED (Replaced by db5874009dd84d4892fccdb3316010c2 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:39.164788 10754 tablet_service.cc:691] Processing DeleteTablet for tablet acb401e40e7049fbbfdbeb40474353d9 with delete_type TABLET_DATA_DELETED (Replaced by d67c8d7da92f4f10b75858f21d80f8c4 at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:39.166072 10754 tablet_service.cc:691] Processing DeleteTablet for tablet 0def0264c0db4c438e16c22b23a0b43a with delete_type TABLET_DATA_DELETED (Replaced by 95f15519e52a48edab3b765ae3a8e93d at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:39.171495 10754 tablet_service.cc:691] Processing DeleteTablet for tablet de37a68992504da8a4640f8ab4653544 with delete_type TABLET_DATA_DELETED (Replaced by 012b21c0f95e4dd594076018fd6fdb7e at 2017-10-31 23:05:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:39.443528 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 11 times. W1031 23:05:39.481709 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 10 times. W1031 23:05:39.548912 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 213 times. W1031 23:05:39.943666 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 12 times. W1031 23:05:39.981861 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 11 times. W1031 23:05:40.048938 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 214 times. W1031 23:05:40.443732 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 13 times. W1031 23:05:40.481873 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 12 times. W1031 23:05:40.549069 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 215 times. W1031 23:05:40.943744 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 14 times. W1031 23:05:40.981904 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 13 times. W1031 23:05:41.049227 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 216 times. W1031 23:05:41.443795 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 15 times. W1031 23:05:41.482003 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 14 times. W1031 23:05:41.549226 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 217 times. W1031 23:05:41.943974 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 16 times. W1031 23:05:41.982136 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 15 times. W1031 23:05:42.049262 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 218 times. W1031 23:05:42.444013 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 17 times. W1031 23:05:42.482142 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 16 times. W1031 23:05:42.549285 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 219 times. I1031 23:05:42.647505 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:42.647547 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.594s I1031 23:05:42.647608 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:42.648325 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:42.648365 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:42.648610 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:42.648695 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:42.648731 10719 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:42.648999 40743 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 3.389s I1031 23:05:42.649032 40743 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:42.944062 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 18 times. W1031 23:05:42.982189 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 17 times. W1031 23:05:43.049387 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 220 times. W1031 23:05:43.444185 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 19 times. W1031 23:05:43.482334 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 18 times. W1031 23:05:43.549528 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 221 times. W1031 23:05:43.944205 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 20 times. W1031 23:05:43.982275 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 19 times. W1031 23:05:44.049679 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 222 times. W1031 23:05:44.444326 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 21 times. W1031 23:05:44.482415 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 20 times. W1031 23:05:44.549582 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 223 times. W1031 23:05:44.944294 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 22 times. W1031 23:05:44.982393 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 21 times. W1031 23:05:45.049778 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 224 times. W1031 23:05:45.444373 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 23 times. W1031 23:05:45.482553 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 22 times. W1031 23:05:45.549720 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 225 times. W1031 23:05:45.944489 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 24 times. W1031 23:05:45.982589 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 23 times. W1031 23:05:46.049831 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 226 times. W1031 23:05:46.444628 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 25 times. W1031 23:05:46.482657 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 24 times. W1031 23:05:46.549840 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 227 times. W1031 23:05:46.944592 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 26 times. W1031 23:05:46.982789 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 25 times. W1031 23:05:47.049968 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 228 times. W1031 23:05:47.444689 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 27 times. W1031 23:05:47.482827 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 26 times. W1031 23:05:47.550030 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 229 times. I1031 23:05:47.848317 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:47.848374 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 5.275s I1031 23:05:47.848450 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:47.849169 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:47.849200 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:47.849428 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:47.849494 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:47.849521 10719 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:47.849818 40753 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 3.204s I1031 23:05:47.849846 40753 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:47.944656 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 28 times. W1031 23:05:47.982882 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 27 times. W1031 23:05:48.050094 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 230 times. W1031 23:05:48.444789 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 29 times. W1031 23:05:48.483041 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 28 times. W1031 23:05:48.550144 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 231 times. W1031 23:05:48.944883 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 30 times. W1031 23:05:48.983145 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 29 times. I1031 23:05:49.018596 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:49.018635 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 14.438s I1031 23:05:49.018692 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:49.019214 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:49.019245 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:49.019482 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:49.019518 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:49.019551 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:49.019690 40756 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 2.867s I1031 23:05:49.019711 40756 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:49.050222 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 232 times. W1031 23:05:49.444994 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 31 times. W1031 23:05:49.483217 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 30 times. W1031 23:05:49.550310 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 233 times. W1031 23:05:49.944975 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 32 times. W1031 23:05:49.983191 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 31 times. W1031 23:05:50.050462 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 234 times. W1031 23:05:50.445042 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 33 times. W1031 23:05:50.483322 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 32 times. W1031 23:05:50.550457 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 235 times. W1031 23:05:50.945107 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 34 times. W1031 23:05:50.983381 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 33 times. W1031 23:05:51.050577 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 236 times. W1031 23:05:51.445773 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 35 times. W1031 23:05:51.483381 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 34 times. W1031 23:05:51.550614 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 237 times. W1031 23:05:51.945245 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 36 times. W1031 23:05:51.983474 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 35 times. W1031 23:05:52.050637 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 238 times. W1031 23:05:52.445417 10718 consensus_peers.cc:357] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 625f49b9554549e6a5c3a92b76e576cf (c3-kudu-tst-st10.bj:18700): Couldn't send request to peer 625f49b9554549e6a5c3a92b76e576cf for tablet eeb3a3cb417b466abffd16749d165a61. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 37 times. W1031 23:05:52.483542 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 36 times. I1031 23:05:52.548722 10754 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:52.548769 10754 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: f2af422919fd4560bd6441509a68d64b I1031 23:05:52.549315 10773 tablet_service.cc:650] Processing CreateTablet for tablet a17da97943904920814af1508905ecfc (table=default.timeout_tst [id=cdcb3e60c1314776843d856365535d49]), partition=HASH (id) PARTITION 0, RANGE (date) PARTITION 20171028 <= VALUES < 20171029 W1031 23:05:52.550705 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 239 times. I1031 23:05:52.606356 10773 ts_tablet_manager.cc:862] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: Registered tablet (data state: TABLET_DATA_READY) I1031 23:05:52.606709 40778 ts_tablet_manager.cc:723] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrapping tablet I1031 23:05:52.607266 40778 tablet_bootstrap.cc:382] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: Bootstrap starting. I1031 23:05:52.607436 40778 tablet_bootstrap.cc:541] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent opening tablet: real 0.000s user 0.000s sys 0.000s I1031 23:05:52.607584 40778 tablet_bootstrap.cc:484] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: No blocks or log segments found. Creating new log. I1031 23:05:52.608942 40778 tablet_bootstrap.cc:382] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: No bootstrap required, opened a new log I1031 23:05:52.608973 40778 ts_tablet_manager.cc:728] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s I1031 23:05:52.609833 40778 raft_consensus.cc:288] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending transactions. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } I1031 23:05:52.609856 40778 raft_consensus.cc:319] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely I1031 23:05:52.609866 40778 raft_consensus.cc:548] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: FOLLOWER I1031 23:05:52.609882 40778 consensus_queue.cc:176] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 0, Majority size: -1, State: 1, Mode: NON_LEADER I1031 23:05:52.813930 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:52.813992 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 8.185s I1031 23:05:52.814055 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:52.814784 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:52.814815 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:52.815038 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:52.815071 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:52.815095 10719 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:52.815805 40783 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 4.283s I1031 23:05:52.815838 40783 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:52.983642 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 37 times. I1031 23:05:52.986106 40661 consensus_queue.cc:695] T eeb3a3cb417b466abffd16749d165a61 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 625f49b9554549e6a5c3a92b76e576cf, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false W1031 23:05:53.050884 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 240 times. I1031 23:05:53.195051 40781 raft_consensus.cc:411] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:53.195147 40781 raft_consensus.cc:2162] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.600s I1031 23:05:53.195246 40781 raft_consensus.cc:435] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } I1031 23:05:53.195950 40781 leader_election.cc:216] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:53.196007 40781 leader_election.cc:216] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 044f1da9a27c46acb82b1386f829f4dc W1031 23:05:53.196286 10719 leader_election.cc:278] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:53.196362 10720 leader_election.cc:348] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Vote granted by peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:05:53.196385 10720 leader_election.cc:243] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. I1031 23:05:53.196461 40782 raft_consensus.cc:2162] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.937s I1031 23:05:53.196493 40782 raft_consensus.cc:1971] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election won for term 1 I1031 23:05:53.196512 40782 raft_consensus.cc:411] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout) I1031 23:05:53.196533 40782 raft_consensus.cc:2162] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.575s I1031 23:05:53.196550 40782 raft_consensus.cc:2206] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Advancing to term 1 I1031 23:05:53.197500 40782 raft_consensus.cc:435] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } I1031 23:05:53.198256 40782 leader_election.cc:216] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:53.198300 40782 leader_election.cc:216] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Requesting vote from peer 044f1da9a27c46acb82b1386f829f4dc W1031 23:05:53.198530 10719 leader_election.cc:278] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:53.199213 10720 leader_election.cc:348] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Vote granted by peer 044f1da9a27c46acb82b1386f829f4dc I1031 23:05:53.199230 10720 leader_election.cc:243] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. I1031 23:05:53.199293 40782 raft_consensus.cc:2162] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.870s I1031 23:05:53.199312 40782 raft_consensus.cc:1971] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 FOLLOWER]: Leader election won for term 1 I1031 23:05:53.199575 40782 raft_consensus.cc:518] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [term 1 LEADER]: Becoming Leader. State: Replica: 0a8bbcbbb5a6459aa5258464755db3bb, State: 1, Role: LEADER I1031 23:05:53.199643 40782 consensus_queue.cc:158] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Current term: 1, Majority size: 2, State: 1, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "044f1da9a27c46acb82b1386f829f4dc" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st03.bj" port: 18700 } } W1031 23:05:53.483642 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 38 times. W1031 23:05:53.550820 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 241 times. W1031 23:05:53.700812 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 1 times. I1031 23:05:53.701377 40785 consensus_queue.cc:695] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 044f1da9a27c46acb82b1386f829f4dc, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:05:53.969990 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:53.970063 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 12.415s I1031 23:05:53.970155 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:53.970779 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:53.970813 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:53.971060 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:53.971156 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:53.971182 10719 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:53.971429 40789 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 14.262s I1031 23:05:53.971452 40789 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:53.983603 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 39 times. W1031 23:05:54.050950 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 242 times. W1031 23:05:54.200870 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 2 times. W1031 23:05:54.483897 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 40 times. W1031 23:05:54.551084 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 243 times. W1031 23:05:54.643147 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 3 times. W1031 23:05:54.983817 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 41 times. W1031 23:05:55.051026 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 244 times. W1031 23:05:55.143013 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 4 times. W1031 23:05:55.483887 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 42 times. W1031 23:05:55.551118 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 245 times. W1031 23:05:55.643172 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 5 times. W1031 23:05:55.983965 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 43 times. W1031 23:05:56.051142 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 246 times. W1031 23:05:56.143224 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 6 times. W1031 23:05:56.484123 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 44 times. W1031 23:05:56.551304 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 247 times. W1031 23:05:56.643282 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 7 times. W1031 23:05:56.984067 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 45 times. W1031 23:05:57.051381 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 248 times. W1031 23:05:57.143323 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 8 times. W1031 23:05:57.484205 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 46 times. W1031 23:05:57.551434 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 249 times. W1031 23:05:57.643394 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 9 times. W1031 23:05:57.984254 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 47 times. W1031 23:05:58.051471 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 250 times. W1031 23:05:58.143563 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 10 times. I1031 23:05:58.247907 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 802f72af09384e95acd389f46a1d9c84 with delete_type TABLET_DATA_DELETED (Replaced by 721e8a57234447eabda00f5df65bd574 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:58.261368 10773 tablet_service.cc:691] Processing DeleteTablet for tablet c94f26e6a0ea4bea82140a0877c0f226 with delete_type TABLET_DATA_DELETED (Replaced by 0c2abca80c8f4fdaa788afd6a931df9d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:58.268779 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 117cd74bcdab4942901acb0005cdd702 with delete_type TABLET_DATA_DELETED (Replaced by acb401e40e7049fbbfdbeb40474353d9 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:58.289187 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 46fc46e028434bc69f467922a9a59101 with delete_type TABLET_DATA_DELETED (Replaced by 7d1359dc00a24df68d41c2447e367f1d at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:58.299104 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 582c1281bf5147b2bd40c9a950a1152f with delete_type TABLET_DATA_DELETED (Replaced by 0def0264c0db4c438e16c22b23a0b43a at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:58.329939 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 8ed42ada2d6846f3b3b70c9d9c13c986 with delete_type TABLET_DATA_DELETED (Replaced by de37a68992504da8a4640f8ab4653544 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:58.358971 10773 tablet_service.cc:691] Processing DeleteTablet for tablet b758ec3274c84dcbb110cf5aa3dca9aa with delete_type TABLET_DATA_DELETED (Replaced by 8356b43879104775b9a8c5a9e546e85b at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:05:58.372417 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 4b81a7f36c9a4b24a3294e39fdf19d9d with delete_type TABLET_DATA_DELETED (Replaced by 1cebccaea10d49478c004cca39676556 at 2017-10-31 23:04:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:05:58.484333 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 48 times. W1031 23:05:58.551517 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 251 times. W1031 23:05:58.643587 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 11 times. W1031 23:05:58.990276 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 49 times. I1031 23:05:59.043334 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:05:59.043385 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 10.331s I1031 23:05:59.043436 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:05:59.044015 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:05:59.044045 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:05:59.044319 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:05:59.044374 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:05:59.044401 10719 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:05:59.044675 40795 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 6.800s I1031 23:05:59.044698 40795 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:05:59.051626 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 252 times. W1031 23:05:59.143669 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 12 times. W1031 23:05:59.490345 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 50 times. W1031 23:05:59.551748 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 253 times. W1031 23:05:59.643826 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 13 times. W1031 23:05:59.990355 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 51 times. W1031 23:06:00.051770 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 254 times. W1031 23:06:00.143781 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 14 times. W1031 23:06:00.490427 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 52 times. W1031 23:06:00.551810 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 255 times. W1031 23:06:00.643923 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 15 times. W1031 23:06:00.990542 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 53 times. W1031 23:06:01.051939 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 256 times. W1031 23:06:01.143942 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 16 times. W1031 23:06:01.490607 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 54 times. W1031 23:06:01.551986 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 257 times. W1031 23:06:01.643982 10719 consensus_peers.cc:357] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 193181be29284287b25df2ee92938f0b (c3-kudu-tst-st08.bj:18700): Couldn't send request to peer 193181be29284287b25df2ee92938f0b for tablet a17da97943904920814af1508905ecfc. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 17 times. W1031 23:06:01.990706 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 55 times. W1031 23:06:02.052068 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 258 times. I1031 23:06:02.164898 40782 consensus_queue.cc:695] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 193181be29284287b25df2ee92938f0b, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false W1031 23:06:02.490769 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 56 times. W1031 23:06:02.552126 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 259 times. W1031 23:06:02.990823 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 57 times. W1031 23:06:03.052274 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 260 times. W1031 23:06:03.490931 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 58 times. W1031 23:06:03.552294 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 261 times. W1031 23:06:03.990993 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 59 times. W1031 23:06:04.052413 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 262 times. W1031 23:06:04.491044 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 60 times. W1031 23:06:04.552397 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 263 times. W1031 23:06:04.991070 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 61 times. W1031 23:06:05.052470 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 264 times. W1031 23:06:05.491194 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 62 times. W1031 23:06:05.552563 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 265 times. W1031 23:06:05.991220 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 63 times. W1031 23:06:06.052660 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 266 times. W1031 23:06:06.491329 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 64 times. W1031 23:06:06.552753 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 267 times. W1031 23:06:06.991331 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 65 times. W1031 23:06:07.052806 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 268 times. I1031 23:06:07.417273 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:06:07.417335 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 3.697s I1031 23:06:07.417418 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:06:07.418040 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:06:07.418077 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:06:07.418277 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:06:07.418460 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:06:07.418530 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:06:07.418884 41002 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 16.441s I1031 23:06:07.418910 41002 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:06:07.491451 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 66 times. W1031 23:06:07.552846 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 269 times. W1031 23:06:07.991484 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 67 times. W1031 23:06:08.052944 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 270 times. W1031 23:06:08.491520 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 68 times. W1031 23:06:08.552994 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 271 times. W1031 23:06:08.991668 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 69 times. W1031 23:06:09.053087 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 272 times. W1031 23:06:09.491700 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 70 times. W1031 23:06:09.553180 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 273 times. W1031 23:06:09.991760 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 71 times. I1031 23:06:10.028605 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:06:10.028676 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 13.205s I1031 23:06:10.028780 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:06:10.029458 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:06:10.029508 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:06:10.029755 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:06:10.029803 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:06:10.029837 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:06:10.030151 41007 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 6.242s I1031 23:06:10.030172 41007 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:06:10.053206 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 274 times. W1031 23:06:10.491858 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 72 times. W1031 23:06:10.553280 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 275 times. W1031 23:06:10.991897 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 73 times. W1031 23:06:11.053416 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 276 times. W1031 23:06:11.492039 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 74 times. W1031 23:06:11.553419 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 277 times. W1031 23:06:11.992060 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 75 times. W1031 23:06:12.053488 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 278 times. W1031 23:06:12.492084 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 76 times. W1031 23:06:12.553516 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 279 times. W1031 23:06:12.992197 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 77 times. W1031 23:06:13.053583 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 280 times. W1031 23:06:13.492221 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 78 times. W1031 23:06:13.553694 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 281 times. W1031 23:06:13.992283 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 79 times. W1031 23:06:14.053683 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 282 times. W1031 23:06:14.492384 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 80 times. W1031 23:06:14.553755 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 283 times. W1031 23:06:14.992503 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 81 times. W1031 23:06:15.053872 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 284 times. W1031 23:06:15.492547 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 82 times. W1031 23:06:15.553961 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 285 times. W1031 23:06:15.992507 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 83 times. W1031 23:06:16.053994 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 286 times. W1031 23:06:16.492663 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 84 times. W1031 23:06:16.554096 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 287 times. W1031 23:06:16.992744 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 85 times. W1031 23:06:17.054162 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 288 times. W1031 23:06:17.492841 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 86 times. W1031 23:06:17.554211 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 289 times. W1031 23:06:17.992921 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 87 times. W1031 23:06:18.054309 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 290 times. I1031 23:06:18.166505 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:06:18.166548 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 7.783s I1031 23:06:18.166601 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:06:18.167150 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:06:18.167176 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:06:18.167368 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:06:18.167426 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:06:18.167464 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:06:18.167775 41042 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 17.797s I1031 23:06:18.167796 41042 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:06:18.492902 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 88 times. W1031 23:06:18.554316 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 291 times. W1031 23:06:18.993052 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 89 times. W1031 23:06:19.054392 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 292 times. W1031 23:06:19.493057 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 90 times. W1031 23:06:19.554496 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 293 times. W1031 23:06:19.993124 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 91 times. W1031 23:06:20.054672 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 294 times. W1031 23:06:20.493219 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 92 times. W1031 23:06:20.554749 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 295 times. W1031 23:06:20.993348 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 93 times. W1031 23:06:21.054718 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 296 times. W1031 23:06:21.493321 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 94 times. W1031 23:06:21.554790 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 297 times. W1031 23:06:21.993404 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 95 times. W1031 23:06:22.054869 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 298 times. W1031 23:06:22.493528 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 96 times. W1031 23:06:22.554975 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 299 times. I1031 23:06:22.591167 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:22.638787 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:22.725255 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:22.861974 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:06:22.993566 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 97 times. W1031 23:06:23.054998 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 300 times. I1031 23:06:23.131274 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:06:23.493623 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 98 times. W1031 23:06:23.555064 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 301 times. I1031 23:06:23.677713 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:06:23.993688 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 99 times. W1031 23:06:24.055142 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 302 times. W1031 23:06:24.493753 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 100 times. W1031 23:06:24.555233 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 303 times. I1031 23:06:24.726461 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:06:24.993830 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 101 times. W1031 23:06:25.055320 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 304 times. W1031 23:06:25.493891 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 102 times. I1031 23:06:25.538655 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:06:25.538708 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 18.822s I1031 23:06:25.538766 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:06:25.539324 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:06:25.539362 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:06:25.539615 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:06:25.539654 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found I1031 23:06:25.539672 10719 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:06:25.539988 41064 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 8.991s I1031 23:06:25.540014 41064 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:06:25.555315 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 305 times. W1031 23:06:25.993938 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 103 times. W1031 23:06:26.055397 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 306 times. W1031 23:06:26.494069 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 104 times. W1031 23:06:26.555567 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 307 times. I1031 23:06:26.800562 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:06:26.994114 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 105 times. W1031 23:06:27.055534 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 308 times. W1031 23:06:27.494215 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 106 times. W1031 23:06:27.555593 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 309 times. W1031 23:06:27.994197 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 107 times. W1031 23:06:28.055670 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 310 times. I1031 23:06:28.286847 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 45e2cfbf12734183aa63ce7e9e78c541 with delete_type TABLET_DATA_DELETED (Replaced by 991306ec7acb449c89e5dd29d90e8d55 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.289237 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 812b248175004ae8992ff2660fc9d520 with delete_type TABLET_DATA_DELETED (Replaced by 8ed42ada2d6846f3b3b70c9d9c13c986 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.292598 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 5110f0095bd94f67abf0aad906332efe with delete_type TABLET_DATA_DELETED (Replaced by bff7dab34ccd4b42842077e82106f6fb at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.297020 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 69b4b5f037014f10bfab077f3f98ca49 with delete_type TABLET_DATA_DELETED (Replaced by b758ec3274c84dcbb110cf5aa3dca9aa at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.319942 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 8182189275034354a3c350318114c7e4 with delete_type TABLET_DATA_DELETED (Replaced by 9710e0187e434acfa54ca8f8504434ed at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.332307 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 6a813ca355f749b6b31a773aa0f2ac7a with delete_type TABLET_DATA_DELETED (Replaced by 582c1281bf5147b2bd40c9a950a1152f at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.354121 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 388697c0c21a4b158a92f2b292b8012f with delete_type TABLET_DATA_DELETED (Replaced by 0ea66c65e2d4445eb9cc83ba7ee1448b at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.375488 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f49a4f915ac048b096fcb5f4878f436d with delete_type TABLET_DATA_DELETED (Replaced by 56155a2adc24450d89e338cc955bfa18 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.377851 10773 tablet_service.cc:691] Processing DeleteTablet for tablet bd6a9e583cad413ca558178148ac7f85 with delete_type TABLET_DATA_DELETED (Replaced by 117cd74bcdab4942901acb0005cdd702 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.426795 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 I1031 23:06:28.426834 10773 tablet_peer.cc:221] Initiating TabletPeer shutdown for tablet: 27f651ff46ca4722b064a85e795dcaa4 W1031 23:06:28.494335 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 108 times. W1031 23:06:28.555687 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 311 times. W1031 23:06:28.994371 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 109 times. W1031 23:06:29.055804 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 312 times. W1031 23:06:29.494446 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 110 times. W1031 23:06:29.555842 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 313 times. W1031 23:06:29.994570 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 111 times. W1031 23:06:30.055971 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 314 times. W1031 23:06:30.494599 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 112 times. W1031 23:06:30.555970 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 315 times. I1031 23:06:30.916837 10758 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:06:30.994654 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 113 times. W1031 23:06:31.056083 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 316 times. W1031 23:06:31.494767 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 114 times. W1031 23:06:31.556129 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 317 times. W1031 23:06:31.994810 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 115 times. W1031 23:06:32.056196 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 318 times. W1031 23:06:32.494886 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 116 times. W1031 23:06:32.556227 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 319 times. W1031 23:06:32.994967 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 117 times. W1031 23:06:33.056239 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 320 times. W1031 23:06:33.494992 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 118 times. W1031 23:06:33.556380 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 321 times. W1031 23:06:33.995085 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 119 times. W1031 23:06:34.056398 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 322 times. W1031 23:06:34.495187 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 120 times. W1031 23:06:34.556489 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 323 times. W1031 23:06:34.995295 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 121 times. W1031 23:06:35.056574 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 324 times. W1031 23:06:35.495355 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 122 times. W1031 23:06:35.556589 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 325 times. W1031 23:06:35.995488 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 123 times. W1031 23:06:36.056761 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 326 times. I1031 23:06:36.095892 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:06:36.095939 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 14.562s I1031 23:06:36.095990 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:06:36.096647 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:06:36.096684 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:06:36.096968 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:06:36.097023 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found I1031 23:06:36.097050 10719 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:06:36.097311 41102 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 7.118s I1031 23:06:36.097333 41102 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:06:36.495481 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 124 times. W1031 23:06:36.556792 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 327 times. W1031 23:06:36.995726 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 125 times. W1031 23:06:37.056892 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 328 times. W1031 23:06:37.495610 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 126 times. W1031 23:06:37.556915 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 329 times. I1031 23:06:37.862576 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:06:37.862624 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.564s I1031 23:06:37.862685 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:06:37.863255 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:06:37.863286 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:06:37.863538 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:06:37.863592 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:06:37.863632 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:06:37.863946 41106 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 8.092s I1031 23:06:37.863968 41106 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:06:37.995687 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 127 times. W1031 23:06:38.056978 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 330 times. W1031 23:06:38.495772 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 128 times. W1031 23:06:38.557057 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 331 times. W1031 23:06:38.995812 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 129 times. W1031 23:06:39.057090 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 332 times. I1031 23:06:39.119572 10758 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:06:39.495909 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 130 times. W1031 23:06:39.557243 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 333 times. W1031 23:06:39.995924 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 131 times. W1031 23:06:40.057200 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 334 times. W1031 23:06:40.496033 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 132 times. W1031 23:06:40.557299 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 335 times. W1031 23:06:40.996134 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 133 times. W1031 23:06:41.057394 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 336 times. W1031 23:06:41.496168 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 134 times. W1031 23:06:41.557412 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 337 times. W1031 23:06:41.996219 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 135 times. W1031 23:06:42.057494 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 338 times. W1031 23:06:42.496316 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 136 times. W1031 23:06:42.557569 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 339 times. W1031 23:06:42.996361 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 137 times. W1031 23:06:43.057667 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 340 times. W1031 23:06:43.496450 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 138 times. W1031 23:06:43.557694 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 341 times. W1031 23:06:43.996527 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 139 times. W1031 23:06:44.057826 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 342 times. W1031 23:06:44.496562 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 140 times. W1031 23:06:44.557869 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 343 times. W1031 23:06:44.996645 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 141 times. W1031 23:06:45.058022 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 344 times. I1031 23:06:45.491061 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:06:45.491134 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 1.582s I1031 23:06:45.491226 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:06:45.491889 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:06:45.491921 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:06:45.492130 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:06:45.492204 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:06:45.492251 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:06:45.492612 41133 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 18.969s I1031 23:06:45.492645 41133 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:06:45.496696 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 142 times. W1031 23:06:45.557934 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 345 times. W1031 23:06:45.996904 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 143 times. W1031 23:06:46.058030 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 346 times. W1031 23:06:46.496872 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 144 times. W1031 23:06:46.558117 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 347 times. W1031 23:06:46.996938 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 145 times. W1031 23:06:47.058194 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 348 times. W1031 23:06:47.497035 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 146 times. W1031 23:06:47.558334 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 349 times. I1031 23:06:47.596565 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:06:47.596623 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 10.189s I1031 23:06:47.596698 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:06:47.597362 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:06:47.597399 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:06:47.597661 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:06:47.597721 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found I1031 23:06:47.597757 10719 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:06:47.598084 41139 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 18.170s I1031 23:06:47.598114 41139 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:06:47.997099 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 147 times. W1031 23:06:48.058398 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 350 times. W1031 23:06:48.497120 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 148 times. W1031 23:06:48.558411 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 351 times. W1031 23:06:48.997191 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 149 times. W1031 23:06:49.058517 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 352 times. W1031 23:06:49.497290 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 150 times. W1031 23:06:49.558581 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 353 times. W1031 23:06:49.997440 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 151 times. W1031 23:06:50.058619 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 354 times. W1031 23:06:50.497376 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 152 times. W1031 23:06:50.558683 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 355 times. W1031 23:06:50.997418 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 153 times. W1031 23:06:51.058735 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 356 times. W1031 23:06:51.497579 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 154 times. W1031 23:06:51.558852 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 357 times. W1031 23:06:51.997566 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 155 times. W1031 23:06:52.058837 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 358 times. W1031 23:06:52.497659 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 156 times. W1031 23:06:52.558915 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 359 times. W1031 23:06:52.997719 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 157 times. W1031 23:06:53.059044 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 360 times. W1031 23:06:53.497753 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 158 times. W1031 23:06:53.559161 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 361 times. W1031 23:06:53.997881 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 159 times. W1031 23:06:54.059213 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 362 times. W1031 23:06:54.497922 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 160 times. W1031 23:06:54.559259 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 363 times. W1031 23:06:54.998005 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 161 times. W1031 23:06:55.059283 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 364 times. W1031 23:06:55.498081 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 162 times. I1031 23:06:55.520758 10758 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:06:55.559331 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 365 times. W1031 23:06:55.998100 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 163 times. W1031 23:06:56.059389 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 366 times. W1031 23:06:56.498178 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 164 times. W1031 23:06:56.559411 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 367 times. W1031 23:06:56.998230 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 165 times. W1031 23:06:57.059515 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 368 times. W1031 23:06:57.498391 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 166 times. W1031 23:06:57.559607 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 369 times. W1031 23:06:57.998477 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 167 times. W1031 23:06:58.059698 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 370 times. W1031 23:06:58.498420 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 168 times. W1031 23:06:58.559746 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 371 times. W1031 23:06:58.998477 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 169 times. W1031 23:06:59.059845 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 372 times. W1031 23:06:59.498620 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 170 times. W1031 23:06:59.559991 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 373 times. W1031 23:06:59.998625 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 171 times. W1031 23:07:00.059945 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 374 times. W1031 23:07:00.498674 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 172 times. W1031 23:07:00.560040 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 375 times. W1031 23:07:00.998845 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 173 times. W1031 23:07:01.060106 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 376 times. W1031 23:07:01.498893 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 174 times. W1031 23:07:01.560132 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 377 times. W1031 23:07:01.998930 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 175 times. W1031 23:07:02.060232 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 378 times. W1031 23:07:02.499119 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 176 times. W1031 23:07:02.560276 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 379 times. W1031 23:07:02.999094 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 177 times. W1031 23:07:03.060405 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 380 times. W1031 23:07:03.499145 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 178 times. W1031 23:07:03.560467 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 381 times. W1031 23:07:03.999344 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 179 times. W1031 23:07:04.060505 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 382 times. W1031 23:07:04.499285 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 180 times. W1031 23:07:04.560632 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 383 times. W1031 23:07:04.999331 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 181 times. W1031 23:07:05.060575 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 384 times. W1031 23:07:05.499445 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 182 times. W1031 23:07:05.560679 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 385 times. W1031 23:07:05.999505 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 183 times. W1031 23:07:06.060845 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 386 times. I1031 23:07:06.130656 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:06.130709 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 4.348s I1031 23:07:06.130784 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:06.131438 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:06.131474 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:06.131675 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:06.131729 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:06.131750 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:06.132016 41408 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 12.664s I1031 23:07:06.132058 41408 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:06.499631 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 184 times. W1031 23:07:06.560878 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 387 times. W1031 23:07:06.999655 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 185 times. W1031 23:07:07.060964 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 388 times. I1031 23:07:07.398680 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:07.398749 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 15.437s I1031 23:07:07.398833 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:07.399451 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:07.399487 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:07.399780 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:07.399844 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:07.399870 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:07.400220 41410 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 18.300s I1031 23:07:07.400240 41410 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:07.499739 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 186 times. W1031 23:07:07.560981 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 389 times. W1031 23:07:07.999827 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 187 times. W1031 23:07:08.061081 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 390 times. W1031 23:07:08.499889 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 188 times. W1031 23:07:08.561110 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 391 times. W1031 23:07:08.999982 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 189 times. W1031 23:07:09.061163 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 392 times. W1031 23:07:09.500082 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 190 times. W1031 23:07:09.561295 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 393 times. W1031 23:07:10.000121 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 191 times. W1031 23:07:10.061269 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 394 times. W1031 23:07:10.500161 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 192 times. W1031 23:07:10.561374 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 395 times. W1031 23:07:11.000217 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 193 times. W1031 23:07:11.061509 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 396 times. W1031 23:07:11.500308 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 194 times. W1031 23:07:11.561499 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 397 times. W1031 23:07:12.000435 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 195 times. W1031 23:07:12.061566 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 398 times. W1031 23:07:12.500433 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 196 times. W1031 23:07:12.561637 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 399 times. W1031 23:07:13.000520 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 197 times. W1031 23:07:13.061664 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 400 times. W1031 23:07:13.500587 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 198 times. W1031 23:07:13.561759 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 401 times. W1031 23:07:14.000653 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 199 times. W1031 23:07:14.061800 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 402 times. W1031 23:07:14.500727 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 200 times. W1031 23:07:14.561930 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 403 times. W1031 23:07:15.000895 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 201 times. W1031 23:07:15.061939 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 404 times. W1031 23:07:15.500892 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 202 times. W1031 23:07:15.562018 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 405 times. W1031 23:07:16.000905 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 203 times. W1031 23:07:16.062109 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 406 times. W1031 23:07:16.501036 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 204 times. W1031 23:07:16.562156 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 407 times. W1031 23:07:17.001011 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 205 times. W1031 23:07:17.062335 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 408 times. W1031 23:07:17.501164 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 206 times. W1031 23:07:17.562440 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 409 times. W1031 23:07:18.001181 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 207 times. W1031 23:07:18.062423 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 410 times. W1031 23:07:18.501379 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 208 times. W1031 23:07:18.562572 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 411 times. W1031 23:07:19.001338 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 209 times. W1031 23:07:19.062599 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 412 times. W1031 23:07:19.501538 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 210 times. W1031 23:07:19.562759 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 413 times. W1031 23:07:20.001411 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 211 times. W1031 23:07:20.062721 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 414 times. I1031 23:07:20.328591 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:20.328641 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 9.172s I1031 23:07:20.328701 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:20.329380 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:20.329407 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:20.329668 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:20.329728 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:20.329767 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:20.330075 41456 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 8.974s I1031 23:07:20.330093 41456 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:20.501579 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 212 times. W1031 23:07:20.562788 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 415 times. W1031 23:07:21.001682 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 213 times. W1031 23:07:21.062865 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 416 times. W1031 23:07:21.501791 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 214 times. W1031 23:07:21.562975 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 417 times. W1031 23:07:22.001731 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 215 times. W1031 23:07:22.062988 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 418 times. W1031 23:07:22.501898 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 216 times. W1031 23:07:22.563076 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 419 times. W1031 23:07:23.001929 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 217 times. W1031 23:07:23.063194 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 420 times. W1031 23:07:23.501992 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 218 times. W1031 23:07:23.563266 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 421 times. W1031 23:07:24.002104 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 219 times. W1031 23:07:24.063282 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 422 times. W1031 23:07:24.502161 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 220 times. W1031 23:07:24.563392 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 423 times. W1031 23:07:25.002308 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 221 times. W1031 23:07:25.063472 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 424 times. W1031 23:07:25.502274 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 222 times. W1031 23:07:25.563450 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 425 times. W1031 23:07:26.002359 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 223 times. W1031 23:07:26.063555 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 426 times. W1031 23:07:26.502491 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 224 times. W1031 23:07:26.563614 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 427 times. W1031 23:07:27.002549 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 225 times. W1031 23:07:27.063740 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 428 times. I1031 23:07:27.275269 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:27.275317 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 10.006s I1031 23:07:27.275374 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:27.275938 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:27.275965 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:27.276219 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED W1031 23:07:27.276271 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found I1031 23:07:27.276295 10719 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:27.276566 41479 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 9.665s I1031 23:07:27.276590 41479 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:27.502622 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 226 times. W1031 23:07:27.563707 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 429 times. W1031 23:07:28.002609 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 227 times. W1031 23:07:28.063812 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 430 times. I1031 23:07:28.334105 10758 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:07:28.502733 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 228 times. W1031 23:07:28.563841 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 431 times. W1031 23:07:29.002859 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 229 times. W1031 23:07:29.063908 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 432 times. W1031 23:07:29.502888 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 230 times. W1031 23:07:29.564036 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 433 times. W1031 23:07:30.003000 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 231 times. W1031 23:07:30.064059 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 434 times. W1031 23:07:30.502986 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 232 times. W1031 23:07:30.564169 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 435 times. W1031 23:07:31.003185 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 233 times. I1031 23:07:31.058964 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:31.059032 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 8.168s I1031 23:07:31.059131 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:31.059778 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:31.059814 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:31.060019 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:31.060065 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:31.060089 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:31.060408 41484 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 6.498s I1031 23:07:31.060430 41484 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:31.064160 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 436 times. W1031 23:07:31.503183 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 234 times. W1031 23:07:31.564317 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 437 times. W1031 23:07:32.003276 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 235 times. W1031 23:07:32.064350 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 438 times. W1031 23:07:32.503495 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 236 times. W1031 23:07:32.564441 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 439 times. W1031 23:07:33.003402 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 237 times. W1031 23:07:33.064496 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 440 times. W1031 23:07:33.503394 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 238 times. W1031 23:07:33.564579 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 441 times. W1031 23:07:34.003578 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 239 times. W1031 23:07:34.064679 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 442 times. W1031 23:07:34.503535 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 240 times. W1031 23:07:34.564765 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 443 times. W1031 23:07:35.003672 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 241 times. W1031 23:07:35.064766 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 444 times. W1031 23:07:35.503758 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 242 times. W1031 23:07:35.564834 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 445 times. W1031 23:07:36.003975 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 243 times. W1031 23:07:36.064903 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 446 times. W1031 23:07:36.503914 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 244 times. W1031 23:07:36.564957 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 447 times. W1031 23:07:37.003981 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 245 times. W1031 23:07:37.065101 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 448 times. W1031 23:07:37.504040 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 246 times. W1031 23:07:37.565062 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 449 times. W1031 23:07:38.004151 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 247 times. W1031 23:07:38.065199 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 450 times. W1031 23:07:38.504247 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 248 times. W1031 23:07:38.565280 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 451 times. W1031 23:07:39.004289 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 249 times. W1031 23:07:39.065408 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 452 times. I1031 23:07:39.152645 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:39.152698 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 18.886s I1031 23:07:39.152760 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:39.153390 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:39.153419 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:39.153630 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:39.153744 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:39.153789 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:39.154129 41622 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 15.855s I1031 23:07:39.154150 41622 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority I1031 23:07:39.375208 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:39.375278 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 7.288s I1031 23:07:39.375344 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:39.375850 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:39.375875 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:39.376044 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:39.376212 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:39.376252 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:39.376818 41623 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 3.565s I1031 23:07:39.376842 41623 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:39.504308 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 250 times. W1031 23:07:39.565405 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 453 times. W1031 23:07:40.004432 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 251 times. W1031 23:07:40.065397 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 454 times. W1031 23:07:40.504492 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 252 times. W1031 23:07:40.565551 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 455 times. W1031 23:07:41.004532 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 253 times. W1031 23:07:41.065629 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 456 times. W1031 23:07:41.504688 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 254 times. W1031 23:07:41.565671 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 457 times. W1031 23:07:42.004740 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 255 times. W1031 23:07:42.065696 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 458 times. W1031 23:07:42.504726 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 256 times. W1031 23:07:42.565762 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 459 times. W1031 23:07:43.004848 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 257 times. W1031 23:07:43.065848 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 460 times. W1031 23:07:43.504899 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 258 times. W1031 23:07:43.565932 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 461 times. W1031 23:07:44.004978 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 259 times. W1031 23:07:44.065973 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 462 times. W1031 23:07:44.505060 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 260 times. W1031 23:07:44.566002 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 463 times. I1031 23:07:44.606624 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:44.606673 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 10.112s I1031 23:07:44.606740 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:44.607306 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:44.607333 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:44.607517 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:44.607558 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:44.607594 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:44.607903 41644 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 5.001s I1031 23:07:44.607923 41644 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:45.005107 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 261 times. W1031 23:07:45.066114 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 464 times. W1031 23:07:45.505121 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 262 times. W1031 23:07:45.566184 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 465 times. W1031 23:07:46.005189 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 263 times. W1031 23:07:46.066200 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 466 times. W1031 23:07:46.505296 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 264 times. W1031 23:07:46.566287 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 467 times. W1031 23:07:47.005367 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 265 times. W1031 23:07:47.066370 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 468 times. W1031 23:07:47.505426 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 266 times. W1031 23:07:47.566444 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 469 times. W1031 23:07:48.005538 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 267 times. W1031 23:07:48.066495 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 470 times. W1031 23:07:48.505580 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 268 times. W1031 23:07:48.566592 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 471 times. W1031 23:07:49.005769 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 269 times. W1031 23:07:49.066695 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 472 times. W1031 23:07:49.505702 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 270 times. W1031 23:07:49.566709 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 473 times. W1031 23:07:50.005779 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 271 times. W1031 23:07:50.066838 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 474 times. W1031 23:07:50.505816 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 272 times. W1031 23:07:50.566905 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 475 times. W1031 23:07:51.005952 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 273 times. W1031 23:07:51.066937 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 476 times. I1031 23:07:51.391211 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:51.391259 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 5.441s I1031 23:07:51.391317 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:51.391964 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:51.392006 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:51.392231 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:51.392340 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:51.392387 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:51.392722 41657 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 13.002s I1031 23:07:51.392761 41657 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:51.505996 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 274 times. W1031 23:07:51.566996 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 477 times. W1031 23:07:52.006055 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 275 times. W1031 23:07:52.067113 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 478 times. W1031 23:07:52.506085 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 276 times. W1031 23:07:52.567121 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 479 times. W1031 23:07:53.006350 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 277 times. W1031 23:07:53.067236 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 480 times. W1031 23:07:53.506306 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 278 times. W1031 23:07:53.567273 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 481 times. W1031 23:07:54.006319 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 279 times. W1031 23:07:54.067358 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 482 times. W1031 23:07:54.506438 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 280 times. W1031 23:07:54.567492 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 483 times. W1031 23:07:55.006543 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 281 times. W1031 23:07:55.067488 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 484 times. W1031 23:07:55.506621 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 282 times. W1031 23:07:55.567512 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 485 times. W1031 23:07:56.006659 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 283 times. W1031 23:07:56.067627 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 486 times. W1031 23:07:56.506784 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 284 times. W1031 23:07:56.567618 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 487 times. I1031 23:07:56.584461 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:07:56.584513 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 11.601s I1031 23:07:56.584575 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:07:56.585211 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:07:56.585242 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:07:56.585496 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:07:56.585539 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:07:56.585588 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:07:56.585906 41682 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 9.890s I1031 23:07:56.585927 41682 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:07:57.006808 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 285 times. W1031 23:07:57.067754 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 488 times. W1031 23:07:57.506903 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 286 times. W1031 23:07:57.567818 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 489 times. W1031 23:07:58.006976 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 287 times. W1031 23:07:58.067821 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 490 times. I1031 23:07:58.445709 10758 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:07:58.506990 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 288 times. W1031 23:07:58.567968 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 491 times. W1031 23:07:59.007095 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 289 times. W1031 23:07:59.067986 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 492 times. W1031 23:07:59.507177 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 290 times. W1031 23:07:59.568109 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 493 times. W1031 23:08:00.007212 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 291 times. W1031 23:08:00.068137 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 494 times. W1031 23:08:00.507290 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 292 times. W1031 23:08:00.568193 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 495 times. W1031 23:08:01.007402 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 293 times. W1031 23:08:01.068310 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 496 times. W1031 23:08:01.507437 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 294 times. W1031 23:08:01.568373 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 497 times. W1031 23:08:02.007508 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 295 times. W1031 23:08:02.068372 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 498 times. W1031 23:08:02.507580 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 296 times. W1031 23:08:02.568496 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 499 times. W1031 23:08:03.007647 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 297 times. W1031 23:08:03.068545 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 500 times. W1031 23:08:03.507680 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 298 times. W1031 23:08:03.568608 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 501 times. W1031 23:08:04.007769 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 299 times. W1031 23:08:04.068681 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 502 times. W1031 23:08:04.507843 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 300 times. W1031 23:08:04.568732 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 503 times. W1031 23:08:05.007973 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 301 times. W1031 23:08:05.068843 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 504 times. W1031 23:08:05.513386 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 302 times. W1031 23:08:05.568903 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 505 times. W1031 23:08:06.013456 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 303 times. I1031 23:08:06.037040 40663 raft_consensus.cc:411] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:08:06.037091 40663 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 4.085s I1031 23:08:06.037171 40663 raft_consensus.cc:435] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:08:06.037787 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:08:06.037825 40663 leader_election.cc:216] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:08:06.038027 10719 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:08:06.038067 10720 leader_election.cc:278] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:08:06.038107 10720 leader_election.cc:243] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:08:06.038455 41921 raft_consensus.cc:2162] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 16.809s I1031 23:08:06.038483 41921 raft_consensus.cc:1908] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:08:06.068928 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 506 times. W1031 23:08:06.513433 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 304 times. W1031 23:08:06.568955 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 507 times. W1031 23:08:07.013572 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 305 times. W1031 23:08:07.069030 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 508 times. W1031 23:08:07.513782 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 306 times. W1031 23:08:07.569118 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 509 times. W1031 23:08:08.013687 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 307 times. W1031 23:08:08.069182 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 510 times. W1031 23:08:08.513766 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 308 times. W1031 23:08:08.569237 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 511 times. I1031 23:08:08.873817 39731 raft_consensus.cc:411] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout) I1031 23:08:08.873872 39731 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 16.476s I1031 23:08:08.873940 39731 raft_consensus.cc:435] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0a8bbcbbb5a6459aa5258464755db3bb" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st02.bj" port: 18700 } } peers { permanent_uuid: "193181be29284287b25df2ee92938f0b" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st08.bj" port: 18700 } } peers { permanent_uuid: "23f962e4a1744381ad5fa0d2d8b10241" member_type: VOTER last_known_addr { host: "c3-kudu-tst-st07.bj" port: 18700 } } I1031 23:08:08.874505 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 193181be29284287b25df2ee92938f0b I1031 23:08:08.874533 39731 leader_election.cc:216] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Requesting pre-vote from peer 23f962e4a1744381ad5fa0d2d8b10241 W1031 23:08:08.874819 10719 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 193181be29284287b25df2ee92938f0b: Not found: Tablet not found W1031 23:08:08.874939 10720 leader_election.cc:278] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Tablet error from VoteRequest() call to peer 23f962e4a1744381ad5fa0d2d8b10241: Illegal state: Tablet not RUNNING: NOT_STARTED I1031 23:08:08.874977 10720 leader_election.cc:243] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. I1031 23:08:08.875288 41928 raft_consensus.cc:2162] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Snoozing failure detection for election timeout plus an additional 14.922s I1031 23:08:08.875309 41928 raft_consensus.cc:1908] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority W1031 23:08:09.013877 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 309 times. W1031 23:08:09.069314 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 512 times. W1031 23:08:09.513907 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 310 times. W1031 23:08:09.569460 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 513 times. W1031 23:08:10.014027 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 311 times. W1031 23:08:10.069463 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 514 times. W1031 23:08:10.514091 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 312 times. W1031 23:08:10.569577 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 515 times. W1031 23:08:11.014170 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 313 times. W1031 23:08:11.069749 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 516 times. W1031 23:08:11.514214 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 314 times. W1031 23:08:11.569700 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 517 times. I1031 23:08:11.875133 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(f2af422919fd4560bd6441509a68d64b) I1031 23:08:11.875195 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op CompactRowSetsOp(27f651ff46ca4722b064a85e795dcaa4) I1031 23:08:11.875211 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(f2af422919fd4560bd6441509a68d64b) I1031 23:08:11.875274 40778 ts_tablet_manager.cc:755] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: Time spent starting tablet: real 139.266s user 0.000s sys 0.001s I1031 23:08:11.875294 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MinorDeltaCompactionOp(27f651ff46ca4722b064a85e795dcaa4) W1031 23:08:11.875306 40778 ts_tablet_manager.cc:785] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet startup took 139266ms I1031 23:08:11.875318 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(27f651ff46ca4722b064a85e795dcaa4) I1031 23:08:11.875340 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(27f651ff46ca4722b064a85e795dcaa4) I1031 23:08:11.875335 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op MajorDeltaCompactionOp(f2af422919fd4560bd6441509a68d64b) I1031 23:08:11.875366 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(27f651ff46ca4722b064a85e795dcaa4) I1031 23:08:11.875376 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op UndoDeltaBlockGCOp(f2af422919fd4560bd6441509a68d64b) I1031 23:08:11.875385 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(27f651ff46ca4722b064a85e795dcaa4) I1031 23:08:11.875393 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushMRSOp(f2af422919fd4560bd6441509a68d64b) I1031 23:08:11.875401 10773 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(27f651ff46ca4722b064a85e795dcaa4) I1031 23:08:11.875406 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op FlushDeltaMemStoresOp(f2af422919fd4560bd6441509a68d64b) I1031 23:08:11.875412 10773 raft_consensus.cc:1569] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:08:11.875416 10754 maintenance_manager.cc:208] P 0a8bbcbbb5a6459aa5258464755db3bb: Unregistered op LogGCOp(f2af422919fd4560bd6441509a68d64b) W1031 23:08:11.875414 40778 ts_tablet_manager.cc:787] T a17da97943904920814af1508905ecfc P 0a8bbcbbb5a6459aa5258464755db3bb: Trace: 1031 23:05:52.549226 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:05:52.549241 (+ 15us) service_pool.cc:202] Handling call 1031 23:05:52.549337 (+ 96us) ts_tablet_manager.cc:275] Acquired tablet manager lock 1031 23:05:52.549341 (+ 4us) ts_tablet_manager.cc:288] Creating new metadata... 1031 23:05:52.605614 (+ 56273us) tablet_metadata.cc:461] Metadata flushed 1031 23:05:52.606691 (+ 1077us) inbound_call.cc:130] Queueing success response 1031 23:05:52.606742 (+ 51us) ts_tablet_manager.cc:724] Bootstrapping tablet 1031 23:05:52.608997 (+ 2255us) ts_tablet_manager.cc:756] Initializing tablet peer 1031 23:05:52.609023 (+ 26us) tablet_peer.cc:161] Creating consensus instance 1031 23:05:52.609415 (+ 392us) tablet_peer.cc:183] Starting instrumentation 1031 23:05:52.609421 (+ 6us) tablet_peer.cc:188] TabletPeer::Init() finished 1031 23:05:52.609421 (+ 0us) ts_tablet_manager.cc:771] Starting tablet peer 1031 23:05:52.609422 (+ 1us) tablet_peer.cc:195] Starting consensus Metrics: {"fdatasync":1,"fdatasync_us":30575,"mutex_wait_us":139264881,"raft.queue_time_us":377,"raft.run_cpu_time_us":4,"raft.run_wall_time_us":3,"tablet-open.queue_time_us":291,"thread_start_us":998,"threads_started":3} I1031 23:08:11.875424 10773 raft_consensus.cc:1585] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:08:11.875428 10754 raft_consensus.cc:1569] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus shutting down. I1031 23:08:11.875463 10754 raft_consensus.cc:1585] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb [term 0 FOLLOWER]: Raft consensus is shut down! I1031 23:08:11.878120 10754 ts_tablet_manager.cc:1020] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:08:11.878334 10773 ts_tablet_manager.cc:1020] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting tablet data with delete state TABLET_DATA_DELETED I1031 23:08:11.895121 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(037f0f4d2e0f4f52b42d6ef46d250ba8): real 0.020s user 0.020s sys 0.000s I1031 23:08:11.895171 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(037f0f4d2e0f4f52b42d6ef46d250ba8) metrics: {"cfile_init":1,"lbm_read_time_us":14,"lbm_reads_lt_1ms":4} W1031 23:08:12.014333 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 315 times. I1031 23:08:12.036203 10754 ts_tablet_manager.cc:1032] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:08:12.036274 10754 log.cc:876] T f2af422919fd4560bd6441509a68d64b P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/f2af422919fd4560bd6441509a68d64b I1031 23:08:12.036505 10754 consensus_meta.cc:71] T f2af422919fd4560bd6441509a68d64b Deleting consensus metadata I1031 23:08:12.061450 10773 ts_tablet_manager.cc:1032] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Tablet deleted. Last logged OpId: 0.0 I1031 23:08:12.061534 10773 log.cc:876] T 27f651ff46ca4722b064a85e795dcaa4 P 0a8bbcbbb5a6459aa5258464755db3bb: Deleting WAL directory at /home/work/app/kudu/c3prc-kernel65/kudu_tablet_server/hdd1/wals/27f651ff46ca4722b064a85e795dcaa4 W1031 23:08:12.061630 10754 rpcz_store.cc:234] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 429) took 139512ms (client timeout 29999). W1031 23:08:12.061738 10754 rpcz_store.cc:238] Trace: 1031 23:05:52.548665 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:05:52.548679 (+ 14us) service_pool.cc:202] Handling call 1031 23:05:52.548763 (+ 84us) ts_tablet_manager.cc:572] Deleting tablet f2af422919fd4560bd6441509a68d64b 1031 23:05:52.548763 (+ 0us) ts_tablet_manager.cc:580] Acquired tablet manager lock 1031 23:08:11.952121 (+139403358us) tablet_metadata.cc:461] Metadata flushed 1031 23:08:12.036178 (+ 84057us) tablet_metadata.cc:461] Metadata flushed 1031 23:08:12.061608 (+ 25430us) inbound_call.cc:130] Queueing success response Metrics: {"fdatasync":2,"fdatasync_us":89195,"mutex_wait_us":139326448} I1031 23:08:12.061830 10773 consensus_meta.cc:71] T 27f651ff46ca4722b064a85e795dcaa4 Deleting consensus metadata W1031 23:08:12.062074 10773 rpcz_store.cc:234] Call kudu.tserver.TabletServerAdminService.DeleteTablet from 10.108.187.10:43437 (request call id 456) took 103635ms (client timeout 29999). W1031 23:08:12.062283 10773 rpcz_store.cc:238] Trace: 1031 23:06:28.426745 (+ 0us) service_pool.cc:143] Inserting onto call queue 1031 23:06:28.426760 (+ 15us) service_pool.cc:202] Handling call 1031 23:06:28.426828 (+ 68us) ts_tablet_manager.cc:572] Deleting tablet 27f651ff46ca4722b064a85e795dcaa4 1031 23:06:28.426828 (+ 0us) ts_tablet_manager.cc:580] Acquired tablet manager lock 1031 23:08:11.968779 (+103541951us) tablet_metadata.cc:461] Metadata flushed 1031 23:08:12.061423 (+ 92644us) tablet_metadata.cc:461] Metadata flushed 1031 23:08:12.062055 (+ 632us) inbound_call.cc:130] Queueing success response Metrics: {"fdatasync":2,"fdatasync_us":89189,"mutex_wait_us":103448401} W1031 23:08:12.069736 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 518 times. I1031 23:08:12.258626 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(fffdb462a7184409ae79f01145545574): real 0.025s user 0.025s sys 0.000s I1031 23:08:12.258677 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(fffdb462a7184409ae79f01145545574) metrics: {"cfile_init":1,"lbm_read_time_us":17,"lbm_reads_lt_1ms":4} W1031 23:08:12.514374 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 316 times. W1031 23:08:12.569772 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 519 times. I1031 23:08:12.848420 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(8fa7cc8c413f452fb2c45eb13d186632): real 0.258s user 0.178s sys 0.009s I1031 23:08:12.848769 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(8fa7cc8c413f452fb2c45eb13d186632) metrics: {"cfile_init":1,"data dir 3.queue_time_us":245,"data dir 3.run_cpu_time_us":89,"data dir 3.run_wall_time_us":89,"fdatasync":1,"fdatasync_us":47844,"lbm_read_time_us":15,"lbm_reads_lt_1ms":4,"tcmalloc_contention_cycles":72448,"thread_start_us":222,"threads_started":1} W1031 23:08:13.014452 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 317 times. W1031 23:08:13.069814 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 520 times. I1031 23:08:13.508752 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(867ed73769f842bda8756be0b016a15a): real 0.313s user 0.185s sys 0.008s I1031 23:08:13.509094 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(867ed73769f842bda8756be0b016a15a) metrics: {"cfile_init":1,"data dir 7.queue_time_us":198,"data dir 7.run_cpu_time_us":64,"data dir 7.run_wall_time_us":64,"fdatasync":1,"fdatasync_us":105060,"lbm_read_time_us":14,"lbm_reads_lt_1ms":4,"tcmalloc_contention_cycles":61568,"thread_start_us":188,"threads_started":1} W1031 23:08:13.514520 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 318 times. W1031 23:08:13.569913 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 521 times. W1031 23:08:14.014577 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 319 times. W1031 23:08:14.069977 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 522 times. I1031 23:08:14.135411 10686 tablet.cc:911] T 2605bb9cd2fe47fea1ad25d47ab8741d P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering stage 1 (old memrowset already frozen for inserts) I1031 23:08:14.135453 10686 compaction.cc:877] Selected 1 rowsets to compact: I1031 23:08:14.135470 10686 compaction.cc:880] memrowset(current size on disk: ~0 bytes) I1031 23:08:14.135488 10686 tablet.cc:913] T 2605bb9cd2fe47fea1ad25d47ab8741d P 0a8bbcbbb5a6459aa5258464755db3bb: Memstore in-memory size: 3751 bytes I1031 23:08:14.135524 10686 tablet.cc:1174] T 2605bb9cd2fe47fea1ad25d47ab8741d P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6182758215047065601 or (T in {6182758215047065601})}] I1031 23:08:14.135677 10686 multi_column_writer.cc:85] Opened CFile writer for column ts[int32 NOT NULL] I1031 23:08:14.135865 10686 multi_column_writer.cc:85] Opened CFile writer for column id[int32 NOT NULL] I1031 23:08:14.259479 10686 tablet.cc:1259] T 2605bb9cd2fe47fea1ad25d47ab8741d P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 2 (starting to duplicate updates in new rowsets) W1031 23:08:14.514636 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 320 times. W1031 23:08:14.570019 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 523 times. I1031 23:08:14.604056 10686 tablet.cc:1314] T 2605bb9cd2fe47fea1ad25d47ab8741d P 0a8bbcbbb5a6459aa5258464755db3bb: Flush Phase 2: carrying over any updates which arrived during Phase 1 I1031 23:08:14.604115 10686 tablet.cc:1316] T 2605bb9cd2fe47fea1ad25d47ab8741d P 0a8bbcbbb5a6459aa5258464755db3bb: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6182758215047065601 or (T in {6182758215047065601})}] W1031 23:08:15.014639 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 321 times. W1031 23:08:15.070058 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 524 times. I1031 23:08:15.190003 10686 tablet.cc:1358] T 2605bb9cd2fe47fea1ad25d47ab8741d P 0a8bbcbbb5a6459aa5258464755db3bb: Flush successful on 18 rows (4411 bytes) I1031 23:08:15.190100 10686 maintenance_manager.cc:419] Time spent running FlushMRSOp(2605bb9cd2fe47fea1ad25d47ab8741d): real 1.380s user 1.148s sys 0.023s I1031 23:08:15.190254 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: FlushMRSOp(2605bb9cd2fe47fea1ad25d47ab8741d) metrics: {"cfile_init":1,"fdatasync":11,"fdatasync_us":193156,"lbm_read_time_us":17,"lbm_reads_lt_1ms":4,"lbm_write_time_us":183,"lbm_writes_lt_1ms":26,"mutex_wait_us":1,"tcmalloc_contention_cycles":75648} W1031 23:08:15.514816 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 322 times. W1031 23:08:15.570211 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 525 times. W1031 23:08:16.014942 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 323 times. W1031 23:08:16.070288 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 526 times. W1031 23:08:16.515044 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 324 times. W1031 23:08:16.570369 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 527 times. W1031 23:08:17.015137 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 325 times. W1031 23:08:17.070406 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 528 times. W1031 23:08:17.515149 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 326 times. W1031 23:08:17.570518 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 529 times. W1031 23:08:18.015251 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 327 times. W1031 23:08:18.070510 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 530 times. W1031 23:08:18.515317 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 328 times. W1031 23:08:18.570556 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 531 times. W1031 23:08:19.015331 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 329 times. W1031 23:08:19.070616 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 532 times. W1031 23:08:19.515383 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 330 times. W1031 23:08:19.570734 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 533 times. W1031 23:08:20.015432 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 331 times. W1031 23:08:20.070706 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 534 times. W1031 23:08:20.515550 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 332 times. W1031 23:08:20.570789 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 535 times. W1031 23:08:21.015554 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 333 times. W1031 23:08:21.070933 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 536 times. W1031 23:08:21.515691 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 334 times. W1031 23:08:21.570987 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 537 times. W1031 23:08:22.015794 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 335 times. W1031 23:08:22.071033 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 538 times. W1031 23:08:22.515784 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 336 times. W1031 23:08:22.571102 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 539 times. W1031 23:08:23.015928 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 337 times. W1031 23:08:23.071197 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 540 times. W1031 23:08:23.515975 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 338 times. W1031 23:08:23.571266 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 541 times. W1031 23:08:24.016029 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 339 times. W1031 23:08:24.071349 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 542 times. W1031 23:08:24.516124 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 340 times. W1031 23:08:24.571435 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 543 times. W1031 23:08:25.016154 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 341 times. W1031 23:08:25.071470 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 544 times. W1031 23:08:25.516278 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 342 times. W1031 23:08:25.571563 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 545 times. W1031 23:08:26.016295 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 343 times. W1031 23:08:26.071584 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 546 times. W1031 23:08:26.516513 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 344 times. W1031 23:08:26.571673 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 547 times. W1031 23:08:27.016453 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 345 times. W1031 23:08:27.071692 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 548 times. W1031 23:08:27.516491 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 346 times. W1031 23:08:27.571817 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 549 times. W1031 23:08:28.016541 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 347 times. W1031 23:08:28.071823 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 550 times. I1031 23:08:28.363287 10773 tablet_service.cc:691] Processing DeleteTablet for tablet f2af422919fd4560bd6441509a68d64b with delete_type TABLET_DATA_DELETED (Replaced by a17da97943904920814af1508905ecfc at 2017-10-31 23:05:52 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:08:28.516681 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 348 times. W1031 23:08:28.571934 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 551 times. W1031 23:08:29.016758 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 349 times. W1031 23:08:29.072000 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 552 times. W1031 23:08:29.516770 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 350 times. W1031 23:08:29.572065 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 553 times. W1031 23:08:30.016877 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 351 times. W1031 23:08:30.072108 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 554 times. W1031 23:08:30.516918 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 352 times. W1031 23:08:30.572196 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 555 times. W1031 23:08:31.016958 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 353 times. W1031 23:08:31.072233 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 556 times. W1031 23:08:31.517083 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 354 times. W1031 23:08:31.572304 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 557 times. W1031 23:08:32.017171 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 355 times. W1031 23:08:32.072381 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 558 times. W1031 23:08:32.517246 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 356 times. W1031 23:08:32.572425 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 559 times. W1031 23:08:33.017225 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 357 times. W1031 23:08:33.072522 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 560 times. W1031 23:08:33.517263 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 358 times. W1031 23:08:33.572576 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 561 times. W1031 23:08:34.017416 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 359 times. W1031 23:08:34.072666 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 562 times. W1031 23:08:34.517424 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 360 times. W1031 23:08:34.572672 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 563 times. W1031 23:08:35.017534 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 361 times. W1031 23:08:35.072767 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 564 times. W1031 23:08:35.517585 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 362 times. W1031 23:08:35.572809 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 565 times. W1031 23:08:36.017663 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 363 times. W1031 23:08:36.072937 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 566 times. W1031 23:08:36.517763 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 364 times. W1031 23:08:36.572932 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 567 times. W1031 23:08:37.017789 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 365 times. W1031 23:08:37.073012 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 568 times. W1031 23:08:37.517917 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 366 times. W1031 23:08:37.573137 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 569 times. W1031 23:08:38.018009 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 367 times. W1031 23:08:38.073237 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 570 times. W1031 23:08:38.518028 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 368 times. W1031 23:08:38.573252 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 571 times. W1031 23:08:39.018097 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 369 times. W1031 23:08:39.073333 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 572 times. W1031 23:08:39.518208 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 370 times. W1031 23:08:39.573359 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 573 times. W1031 23:08:40.018254 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 371 times. W1031 23:08:40.073395 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 574 times. W1031 23:08:40.518365 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 372 times. W1031 23:08:40.573509 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 575 times. W1031 23:08:41.018407 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 373 times. W1031 23:08:41.073565 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 576 times. W1031 23:08:41.518465 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 374 times. W1031 23:08:41.573634 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 577 times. W1031 23:08:42.018584 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 375 times. W1031 23:08:42.073726 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 578 times. W1031 23:08:42.518606 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 376 times. W1031 23:08:42.573767 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 579 times. W1031 23:08:43.018661 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 377 times. W1031 23:08:43.073938 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 580 times. W1031 23:08:43.518781 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 378 times. W1031 23:08:43.574021 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 581 times. W1031 23:08:44.018856 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 379 times. W1031 23:08:44.074025 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 582 times. W1031 23:08:44.526393 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 380 times. W1031 23:08:44.574093 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 583 times. W1031 23:08:45.026396 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 381 times. W1031 23:08:45.074152 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 584 times. W1031 23:08:45.526537 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 382 times. W1031 23:08:45.574270 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 585 times. W1031 23:08:46.026579 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 383 times. W1031 23:08:46.074273 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 586 times. W1031 23:08:46.526597 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 384 times. W1031 23:08:46.574278 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 587 times. W1031 23:08:47.026806 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 385 times. W1031 23:08:47.074529 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 588 times. W1031 23:08:47.526782 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 386 times. W1031 23:08:47.574450 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 589 times. W1031 23:08:48.026852 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 387 times. W1031 23:08:48.074539 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 590 times. W1031 23:08:48.526983 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 388 times. W1031 23:08:48.574615 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 591 times. W1031 23:08:49.027076 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 389 times. W1031 23:08:49.074712 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 592 times. W1031 23:08:49.527097 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 390 times. W1031 23:08:49.574728 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 593 times. W1031 23:08:50.027143 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 391 times. W1031 23:08:50.074820 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 594 times. W1031 23:08:50.527277 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 392 times. W1031 23:08:50.574884 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 595 times. W1031 23:08:51.027338 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 393 times. W1031 23:08:51.074976 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 596 times. W1031 23:08:51.527386 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 394 times. W1031 23:08:51.575037 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 597 times. W1031 23:08:52.027510 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 395 times. W1031 23:08:52.075122 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 598 times. W1031 23:08:52.527529 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 396 times. W1031 23:08:52.575158 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 599 times. W1031 23:08:53.027587 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 397 times. W1031 23:08:53.075239 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 600 times. W1031 23:08:53.527797 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 398 times. W1031 23:08:53.575366 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 601 times. W1031 23:08:54.027724 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 399 times. W1031 23:08:54.075431 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 602 times. W1031 23:08:54.527830 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 400 times. W1031 23:08:54.575549 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 603 times. W1031 23:08:55.027858 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 401 times. W1031 23:08:55.075562 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 604 times. W1031 23:08:55.527918 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 402 times. W1031 23:08:55.575677 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 605 times. W1031 23:08:56.027967 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 403 times. W1031 23:08:56.075728 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 606 times. W1031 23:08:56.528085 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 404 times. W1031 23:08:56.575867 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 607 times. W1031 23:08:57.028081 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 405 times. W1031 23:08:57.075820 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 608 times. W1031 23:08:57.528211 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 406 times. W1031 23:08:57.575948 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 609 times. W1031 23:08:58.028234 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 407 times. W1031 23:08:58.075963 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 610 times. I1031 23:08:58.465188 10773 tablet_service.cc:691] Processing DeleteTablet for tablet 27f651ff46ca4722b064a85e795dcaa4 with delete_type TABLET_DATA_DELETED (Replaced by 46fc46e028434bc69f467922a9a59101 at 2017-10-31 23:04:22 CST) from {username='work'} at 10.108.187.10:43437 W1031 23:08:58.528360 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 408 times. W1031 23:08:58.576082 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 611 times. W1031 23:08:59.028363 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 409 times. W1031 23:08:59.076050 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 612 times. W1031 23:08:59.528439 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 410 times. W1031 23:08:59.576103 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 613 times. W1031 23:09:00.028537 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 411 times. W1031 23:09:00.076185 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 614 times. W1031 23:09:00.528635 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 412 times. W1031 23:09:00.576244 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 615 times. W1031 23:09:01.028663 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 413 times. W1031 23:09:01.076503 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 616 times. W1031 23:09:01.528719 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 414 times. W1031 23:09:01.576504 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 617 times. W1031 23:09:02.028786 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 415 times. W1031 23:09:02.076612 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 618 times. W1031 23:09:02.528815 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 416 times. W1031 23:09:02.576568 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 619 times. W1031 23:09:03.028867 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 417 times. W1031 23:09:03.076642 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 620 times. W1031 23:09:03.528987 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 418 times. W1031 23:09:03.576695 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 621 times. W1031 23:09:04.029112 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 419 times. W1031 23:09:04.076771 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 622 times. W1031 23:09:04.529081 10720 consensus_peers.cc:357] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 47eeb8448c454968a45bf1a984427b25. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 420 times. W1031 23:09:04.576858 10720 consensus_peers.cc:357] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb -> Peer 23f962e4a1744381ad5fa0d2d8b10241 (c3-kudu-tst-st07.bj:18700): Couldn't send request to peer 23f962e4a1744381ad5fa0d2d8b10241 for tablet 68aa6071fa594e79958cddb8489a1f0c. Error code: TABLET_NOT_RUNNING (12). Status: Illegal state: Tablet not RUNNING: NOT_STARTED. Retrying in the next heartbeat period. Already tried 623 times. I1031 23:09:05.029726 40698 consensus_queue.cc:695] T 47eeb8448c454968a45bf1a984427b25 P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 23f962e4a1744381ad5fa0d2d8b10241, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:09:05.077431 39777 consensus_queue.cc:695] T 68aa6071fa594e79958cddb8489a1f0c P 0a8bbcbbb5a6459aa5258464755db3bb [LEADER]: Connected to new peer: Peer: 23f962e4a1744381ad5fa0d2d8b10241, Is new: false, Last received: 0.0, Next index: 1, Last known committed idx: 0, Last exchange result: ERROR, Needs tablet copy: false I1031 23:10:50.806293 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(2605bb9cd2fe47fea1ad25d47ab8741d): real 0.025s user 0.025s sys 0.000s I1031 23:10:50.806347 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(2605bb9cd2fe47fea1ad25d47ab8741d) metrics: {"cfile_init":1,"lbm_read_time_us":16,"lbm_reads_lt_1ms":4} I1031 23:10:51.518450 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(c62e1dbe04164d22bb6fdf7674bc18c5): real 0.375s user 0.191s sys 0.007s I1031 23:10:51.518757 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(c62e1dbe04164d22bb6fdf7674bc18c5) metrics: {"data dir 0.queue_time_us":276,"data dir 0.run_cpu_time_us":51,"data dir 0.run_wall_time_us":51,"fdatasync":1,"fdatasync_us":161661,"tcmalloc_contention_cycles":163200,"thread_start_us":266,"threads_started":1} I1031 23:10:52.094032 10686 tablet.cc:911] T 785db89de0cc49b0aaa355be33e43195 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering stage 1 (old memrowset already frozen for inserts) I1031 23:10:52.094076 10686 compaction.cc:877] Selected 1 rowsets to compact: I1031 23:10:52.094089 10686 compaction.cc:880] memrowset(current size on disk: ~0 bytes) I1031 23:10:52.094101 10686 tablet.cc:913] T 785db89de0cc49b0aaa355be33e43195 P 0a8bbcbbb5a6459aa5258464755db3bb: Memstore in-memory size: 3751 bytes I1031 23:10:52.094123 10686 tablet.cc:1174] T 785db89de0cc49b0aaa355be33e43195 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6182758950388105216 or (T in {6182758950388105216})}] I1031 23:10:52.094349 10686 multi_column_writer.cc:85] Opened CFile writer for column ts[int32 NOT NULL] I1031 23:10:52.094389 10686 multi_column_writer.cc:85] Opened CFile writer for column id[int32 NOT NULL] I1031 23:10:52.279326 10686 tablet.cc:1259] T 785db89de0cc49b0aaa355be33e43195 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I1031 23:10:52.634325 10686 tablet.cc:1314] T 785db89de0cc49b0aaa355be33e43195 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush Phase 2: carrying over any updates which arrived during Phase 1 I1031 23:10:52.634393 10686 tablet.cc:1316] T 785db89de0cc49b0aaa355be33e43195 P 0a8bbcbbb5a6459aa5258464755db3bb: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6182758950388105216 or (T in {6182758950388105216})}] I1031 23:10:53.252969 10686 tablet.cc:1358] T 785db89de0cc49b0aaa355be33e43195 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush successful on 16 rows (4329 bytes) I1031 23:10:53.253218 10686 maintenance_manager.cc:419] Time spent running FlushMRSOp(785db89de0cc49b0aaa355be33e43195): real 1.501s user 1.200s sys 0.044s I1031 23:10:53.253293 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: FlushMRSOp(785db89de0cc49b0aaa355be33e43195) metrics: {"cfile_init":1,"fdatasync":11,"fdatasync_us":241420,"lbm_read_time_us":9,"lbm_reads_lt_1ms":4,"lbm_write_time_us":167,"lbm_writes_lt_1ms":26,"tcmalloc_contention_cycles":80384} I1031 23:13:30.688496 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(785db89de0cc49b0aaa355be33e43195): real 0.027s user 0.026s sys 0.001s I1031 23:13:30.688561 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(785db89de0cc49b0aaa355be33e43195) metrics: {"cfile_init":1,"lbm_read_time_us":14,"lbm_reads_lt_1ms":4} I1031 23:13:31.358233 10686 tablet.cc:911] T 805311cf0a9549e08dc0767b41b859ec P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering stage 1 (old memrowset already frozen for inserts) I1031 23:13:31.358283 10686 compaction.cc:877] Selected 1 rowsets to compact: I1031 23:13:31.358300 10686 compaction.cc:880] memrowset(current size on disk: ~0 bytes) I1031 23:13:31.358314 10686 tablet.cc:913] T 805311cf0a9549e08dc0767b41b859ec P 0a8bbcbbb5a6459aa5258464755db3bb: Memstore in-memory size: 3751 bytes I1031 23:13:31.358351 10686 tablet.cc:1174] T 805311cf0a9549e08dc0767b41b859ec P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6182759443353628672 or (T in {6182759443353628672})}] I1031 23:13:31.358536 10686 multi_column_writer.cc:85] Opened CFile writer for column ts[int32 NOT NULL] I1031 23:13:31.358785 10686 multi_column_writer.cc:85] Opened CFile writer for column id[int32 NOT NULL] I1031 23:13:31.927572 10686 tablet.cc:1259] T 805311cf0a9549e08dc0767b41b859ec P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I1031 23:13:32.307482 10686 tablet.cc:1314] T 805311cf0a9549e08dc0767b41b859ec P 0a8bbcbbb5a6459aa5258464755db3bb: Flush Phase 2: carrying over any updates which arrived during Phase 1 I1031 23:13:32.307549 10686 tablet.cc:1316] T 805311cf0a9549e08dc0767b41b859ec P 0a8bbcbbb5a6459aa5258464755db3bb: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6182759443353628672 or (T in {6182759443353628672})}] I1031 23:13:32.930008 10686 tablet.cc:1358] T 805311cf0a9549e08dc0767b41b859ec P 0a8bbcbbb5a6459aa5258464755db3bb: Flush successful on 13 rows (4351 bytes) I1031 23:13:32.930119 10686 maintenance_manager.cc:419] Time spent running FlushMRSOp(805311cf0a9549e08dc0767b41b859ec): real 1.907s user 1.236s sys 0.034s I1031 23:13:32.930369 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: FlushMRSOp(805311cf0a9549e08dc0767b41b859ec) metrics: {"cfile_init":1,"fdatasync":11,"fdatasync_us":621704,"lbm_read_time_us":14,"lbm_reads_lt_1ms":4,"lbm_write_time_us":289,"lbm_writes_lt_1ms":26,"tcmalloc_contention_cycles":715520} I1031 23:16:09.929167 10686 tablet.cc:911] T 9d015a51a6ef49bd903db1f74f5872e1 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering stage 1 (old memrowset already frozen for inserts) I1031 23:16:09.929198 10686 compaction.cc:877] Selected 1 rowsets to compact: I1031 23:16:09.929229 10686 compaction.cc:880] memrowset(current size on disk: ~0 bytes) I1031 23:16:09.929240 10686 tablet.cc:913] T 9d015a51a6ef49bd903db1f74f5872e1 P 0a8bbcbbb5a6459aa5258464755db3bb: Memstore in-memory size: 4079383 bytes I1031 23:16:09.929255 10686 tablet.cc:1174] T 9d015a51a6ef49bd903db1f74f5872e1 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6182760323247104000 or (T in {6182760323247104000})}] I1031 23:16:09.929406 10686 multi_column_writer.cc:85] Opened CFile writer for column olap_date[int32 NOT NULL] I1031 23:16:09.929471 10686 multi_column_writer.cc:85] Opened CFile writer for column timestamp[int64 NOT NULL] I1031 23:16:09.929533 10686 multi_column_writer.cc:85] Opened CFile writer for column tweet_id[int64 NOT NULL] I1031 23:16:09.929596 10686 multi_column_writer.cc:85] Opened CFile writer for column created_at[string NULLABLE] I1031 23:16:09.929658 10686 multi_column_writer.cc:85] Opened CFile writer for column text[string NULLABLE] I1031 23:16:09.929702 10686 multi_column_writer.cc:85] Opened CFile writer for column source[string NULLABLE] I1031 23:16:09.929736 10686 multi_column_writer.cc:85] Opened CFile writer for column user_id[int64 NULLABLE] I1031 23:16:09.929782 10686 multi_column_writer.cc:85] Opened CFile writer for column user_name[string NULLABLE] I1031 23:16:09.929822 10686 multi_column_writer.cc:85] Opened CFile writer for column user_location[string NULLABLE] I1031 23:16:09.929852 10686 multi_column_writer.cc:85] Opened CFile writer for column user_followers_count[int32 NULLABLE] I1031 23:16:09.929882 10686 multi_column_writer.cc:85] Opened CFile writer for column user_friends_count[int32 NULLABLE] I1031 23:16:10.202111 10686 tablet.cc:1259] T 9d015a51a6ef49bd903db1f74f5872e1 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I1031 23:16:10.202476 10686 tablet.cc:1314] T 9d015a51a6ef49bd903db1f74f5872e1 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush Phase 2: carrying over any updates which arrived during Phase 1 I1031 23:16:10.202494 10686 tablet.cc:1316] T 9d015a51a6ef49bd903db1f74f5872e1 P 0a8bbcbbb5a6459aa5258464755db3bb: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6182760325707255808 or (T in {6182760325707255808})}] I1031 23:16:10.261270 10686 tablet.cc:1358] T 9d015a51a6ef49bd903db1f74f5872e1 P 0a8bbcbbb5a6459aa5258464755db3bb: Flush successful on 7441 rows (1245009 bytes) I1031 23:16:10.261729 10686 maintenance_manager.cc:419] Time spent running FlushMRSOp(9d015a51a6ef49bd903db1f74f5872e1): real 0.333s user 0.038s sys 0.006s I1031 23:16:10.261787 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: FlushMRSOp(9d015a51a6ef49bd903db1f74f5872e1) metrics: {"cfile_init":1,"data dir 8.queue_time_us":266,"data dir 8.run_cpu_time_us":136,"data dir 8.run_wall_time_us":24224,"fdatasync":29,"fdatasync_us":272178,"lbm_read_time_us":8,"lbm_reads_lt_1ms":4,"lbm_write_time_us":2372,"lbm_writes_lt_1ms":116,"thread_start_us":251,"threads_started":1} I1031 23:16:10.538714 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(9d015a51a6ef49bd903db1f74f5872e1): real 0.000s user 0.000s sys 0.000s I1031 23:16:10.538758 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: UndoDeltaBlockGCOp(9d015a51a6ef49bd903db1f74f5872e1) metrics: {"cfile_init":1,"lbm_read_time_us":12,"lbm_reads_lt_1ms":4} I1031 23:16:11.221354 10686 tablet.cc:911] T 867ed73769f842bda8756be0b016a15a P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering stage 1 (old memrowset already frozen for inserts) I1031 23:16:11.221395 10686 compaction.cc:877] Selected 1 rowsets to compact: I1031 23:16:11.221418 10686 compaction.cc:880] memrowset(current size on disk: ~0 bytes) I1031 23:16:11.221434 10686 tablet.cc:913] T 867ed73769f842bda8756be0b016a15a P 0a8bbcbbb5a6459aa5258464755db3bb: Memstore in-memory size: 3751 bytes I1031 23:16:11.221454 10686 tablet.cc:1174] T 867ed73769f842bda8756be0b016a15a P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 1 (flushing snapshot). Phase 1 snapshot: MvccSnapshot[committed={T|T < 6182760182003662848 or (T in {6182760182003662848})}] I1031 23:16:11.221673 10686 multi_column_writer.cc:85] Opened CFile writer for column ts[int32 NOT NULL] I1031 23:16:11.221712 10686 multi_column_writer.cc:85] Opened CFile writer for column id[int32 NOT NULL] I1031 23:16:11.370271 10686 tablet.cc:1259] T 867ed73769f842bda8756be0b016a15a P 0a8bbcbbb5a6459aa5258464755db3bb: Flush: entering phase 2 (starting to duplicate updates in new rowsets) I1031 23:16:11.716640 10686 tablet.cc:1314] T 867ed73769f842bda8756be0b016a15a P 0a8bbcbbb5a6459aa5258464755db3bb: Flush Phase 2: carrying over any updates which arrived during Phase 1 I1031 23:16:11.716702 10686 tablet.cc:1316] T 867ed73769f842bda8756be0b016a15a P 0a8bbcbbb5a6459aa5258464755db3bb: Phase 2 snapshot: MvccSnapshot[committed={T|T < 6182760330591404032 or (T in {6182760330591404032})}] I1031 23:16:12.293676 10686 tablet.cc:1358] T 867ed73769f842bda8756be0b016a15a P 0a8bbcbbb5a6459aa5258464755db3bb: Flush successful on 19 rows (4395 bytes) I1031 23:16:12.293762 10686 maintenance_manager.cc:419] Time spent running FlushMRSOp(867ed73769f842bda8756be0b016a15a): real 1.392s user 1.149s sys 0.028s I1031 23:16:12.293917 10686 maintenance_manager.cc:425] P 0a8bbcbbb5a6459aa5258464755db3bb: FlushMRSOp(867ed73769f842bda8756be0b016a15a) metrics: {"cfile_init":1,"fdatasync":11,"fdatasync_us":199759,"lbm_read_time_us":12,"lbm_reads_lt_1ms":4,"lbm_write_time_us":153,"lbm_writes_lt_1ms":26,"tcmalloc_contention_cycles":124544} I1031 23:18:48.511956 10686 maintenance_manager.cc:419] Time spent running UndoDeltaBlockGCOp(867ed73769f842bda8756be0b016a15a): real 0.020s user 0.019s sys 0.000s